==================== ISSUES SUMMARY ==================== ========== ODH/RHOAI ISSUES SUMMARY (status, events, highlights only) ========== 132m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 129m Warning Unhealthy pod/kserve-controller-manager-6c8cb86d7f-st4s8 Readiness probe failed: Get "http://10.129.3.45:8081/readyz": dial tcp 10.129.3.45:8081: connect: connection refused 129m Warning FailedToRetrieveImagePullSecret pod/selector-migration-n4q8g Unable to retrieve some image pull secrets (feast-operator-controller-manager-dockercfg-xvxzx); attempting to pull the image may not succeed. 129m Warning Unhealthy pod/model-serving-api-78bbbb9bf8-zmqbh Readiness probe failed: Get "https://10.130.3.13:8443/readyz": context deadline exceeded 129m Warning FailedCreate replicaset/feast-operator-controller-manager-744655f689 Error creating: pods "feast-operator-controller-manager-744655f689-" is forbidden: error looking up service account redhat-ods-applications/feast-operator-controller-manager: serviceaccount "feast-operator-controller-manager" not found 129m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-n6hks MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 129m Warning FailedCreate job/selector-migration Error creating: pods "selector-migration-" is forbidden: error looking up service account redhat-ods-applications/feast-operator-controller-manager: serviceaccount "feast-operator-controller-manager" not found 129m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-mvvkb MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 129m Warning FailedMount pod/odh-model-controller-6b647f87c-g2lwp MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 129m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 129m Warning FailedMount pod/odh-notebook-controller-manager-7f6b58f576-dkjsh MountVolume.SetUp failed for volume "webhook-cert" : secret "odh-notebook-controller-webhook-cert" not found 129m Warning FailedCreate replicaset/notebook-controller-deployment-5d7d79cf5b Error creating: pods "notebook-controller-deployment-5d7d79cf5b-" is forbidden: error looking up service account redhat-ods-applications/notebook-controller-service-account: serviceaccount "notebook-controller-service-account" not found 128m Warning FailedToUpdateEndpointSlices service/ogx-k8s-operator-controller-manager-metrics-service Error updating Endpoint Slices for Service redhat-ods-applications/ogx-k8s-operator-controller-manager-metrics-service: failed to delete ogx-k8s-operator-controller-manager-metrics-service-8sngc EndpointSlice for Service redhat-ods-applications/ogx-k8s-operator-controller-manager-metrics-service: endpointslices.discovery.k8s.io "ogx-k8s-operator-controller-manager-metrics-service-8sngc" not found 128m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 128m Warning CalculateExpectedPodCountFailed poddisruptionbudget/ogx-k8s-operator-controller-manager-pdb Failed to calculate the number of expected pods: found no controllers for pod "ogx-k8s-operator-controller-manager-67866ffc79-n6hks" 128m Warning CalculateExpectedPodCountFailed poddisruptionbudget/ogx-k8s-operator-controller-manager-pdb Failed to calculate the number of expected pods: found no controllers for pod "ogx-k8s-operator-controller-manager-67866ffc79-pdzlm" 127m Warning FailedMount pod/data-science-pipelines-operator-controller-manager-7fc7f762ncnr MountVolume.SetUp failed for volume "config" : configmap "data-science-pipelines-operator-dspo-config" not found 127m Warning FailedCreate replicaset/rhods-dashboard-6b5cc9785c Error creating: pods "rhods-dashboard-6b5cc9785c-" is forbidden: error looking up service account redhat-ods-applications/rhods-dashboard: serviceaccount "rhods-dashboard" not found 127m Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 127m Warning FailedMount pod/kuberay-operator-6b449f854b-hzkwl MountVolume.SetUp failed for volume "cert" : secret "kuberay-webhook-server-cert" not found 126m Warning FailedMount pod/spark-pi-d97ctlmb1jhc7b532630-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-85b93b9f43d82162-conf-map" not found 126m Warning FailedMount pod/spark-pi-d97ctlmb1jhc7b532630-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "spark-pi-d97ctlmb1jhc7b532630-fd5abb9f43d81031-driver-podspec-conf-map" not found 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-26lrs Error: ErrImagePull 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-26lrs Failed to pull image "registry.redhat.io/rhoai/odh-mod-arch-model-registry-rhel9@sha256:1ce99d94d4af0a582e3a98a4e6a7758652a98c2e4dd6fff2380cfdd925a6f5cf": copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/a9/a9d40ceb865f72b9ecca3cc5dca6cac11a6e344d4273c7d2f67ca4d4e1556d9c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260708%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260708T222848Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=0b371b64ab9d8da3792e9bf06f65e6cc14c3115729524771caa6005a42b09dba®ion=us-east-1&namespace=rhoai&username=rhoai+rhoai_devops_konfux_bot&repo_name=odh-mod-arch-model-registry-rhel9&akamai_signature=exp=1783550628~hmac=14eac819a45da307f16ac74985ff394b85b4f99238add35156a120dc9a909fc9": EOF 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-26lrs Error: ImagePullBackOff 124m Warning FailedMount pod/scheduled-spark-pi-d97cu26b1jhc7b532640-1783549734000822948-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-1d7a259f43d95bcf-conf-map" not found 124m Warning FailedMount pod/scheduled-spark-pi-d97cu26b1jhc7b532640-1783549734000822948-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "scheduled-spark-pi-d97cu26b1jhc7b532640-1783549734000822948-f707b19f43d949cd-driver-podspec-conf-map" not found 118m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-86x4p MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 117m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 117m Warning InvalidConfiguration servicemonitor/workload-variant-autoscaler-controller-manager-metrics-monitor ServiceMonitor workload-variant-autoscaler-controller-manager-metrics-monitor was rejected due to invalid configuration: failed to get authorization token of type "": unable to get secret "workload-variant-autoscaler-metrics-reader-token": secrets "workload-variant-autoscaler-metrics-reader-token" not found 114m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 111m Warning FailedMount pod/odh-model-controller-54bf759f64-gt5l2 MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 111m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 110m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 110m Warning FailedCreate replicaset/trustyai-service-operator-controller-manager-6f7f88c874 Error creating: pods "trustyai-service-operator-controller-manager-6f7f88c874-" is forbidden: error looking up service account redhat-ods-applications/trustyai-service-operator-controller-manager: serviceaccount "trustyai-service-operator-controller-manager" not found 110m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 110m Warning FailedToUpdateEndpoint endpoints/mlflow-operator-controller-manager-metrics-service Failed to update endpoint redhat-ods-applications/mlflow-operator-controller-manager-metrics-service: Operation cannot be fulfilled on endpoints "mlflow-operator-controller-manager-metrics-service": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/redhat-ods-applications/mlflow-operator-controller-manager-metrics-service, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 330380f2-ca5a-4f73-9613-db635448e8e7, UID in object meta: 108m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 107m Warning FailedMount pod/odh-model-controller-54bf759f64-2n87k MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 107m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 107m Warning FailedMount pod/kserve-controller-manager-6c8cb86d7f-nrmd4 MountVolume.SetUp failed for volume "cert" : secret "kserve-webhook-server-cert" not found 107m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 105m Warning BackOff pod/maas-controller-656b6867cb-vznfq Back-off restarting failed container manager in pod maas-controller-656b6867cb-vznfq_redhat-ods-applications(17cbcc19-188b-4f00-bbeb-825e0a39a8f1) 100m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 99m Warning FailedMount pod/odh-model-controller-54bf759f64-wgh9l MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 99m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 99m Warning FailedMount pod/model-serving-api-78bbbb9bf8-r7twk MountVolume.SetUp failed for volume "tls-certs" : secret "model-serving-api-tls" not found 99m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 98m Warning FailedMount pod/kserve-controller-manager-6c8cb86d7f-6b97l MountVolume.SetUp failed for volume "cert" : secret "kserve-webhook-server-cert" not found 98m Warning FailedMount pod/odh-model-controller-54bf759f64-bt4fr MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 97m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 97m Warning FailedMount pod/kserve-controller-manager-6c8cb86d7f-rj9zj MountVolume.SetUp failed for volume "cert" : secret "kserve-webhook-server-cert" not found 96m Warning FailedCreate replicaset/trustyai-service-operator-controller-manager-6f7f88c874 Error creating: pods "trustyai-service-operator-controller-manager-6f7f88c874-" is forbidden: error looking up service account redhat-ods-applications/trustyai-service-operator-controller-manager: serviceaccount "trustyai-service-operator-controller-manager" not found 96m Warning FailedMount pod/mlflow-operator-controller-manager-8646f94cf4-4jxqg MountVolume.SetUp failed for volume "metrics-cert" : secret "controller-manager-metrics-tls" not found 96m Warning FailedMount pod/data-science-pipelines-operator-controller-manager-7fc7f76glsfg MountVolume.SetUp failed for volume "config" : configmap "data-science-pipelines-operator-dspo-config" not found 96m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-cq44q MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 96m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-j9r4m MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 96m Warning FailedMount pod/odh-notebook-controller-manager-7f6b58f576-dfxz4 MountVolume.SetUp failed for volume "webhook-cert" : secret "odh-notebook-controller-webhook-cert" not found 96m Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 96m Warning FailedCreate replicaset/notebook-controller-deployment-5d7d79cf5b Error creating: pods "notebook-controller-deployment-5d7d79cf5b-" is forbidden: error looking up service account redhat-ods-applications/notebook-controller-service-account: serviceaccount "notebook-controller-service-account" not found 96m Warning BackOff pod/trustyai-service-operator-controller-manager-6f7f88c874-242cm Back-off restarting failed container manager in pod trustyai-service-operator-controller-manager-6f7f88c874-242cm_redhat-ods-applications(a5408679-a4ef-468f-9f5a-1f7c755f16d0) 96m Warning FailedCreate replicaset/rhods-dashboard-6b5cc9785c Error creating: pods "rhods-dashboard-6b5cc9785c-" is forbidden: error looking up service account redhat-ods-applications/rhods-dashboard: serviceaccount "rhods-dashboard" not found 96m Warning FailedMount pod/kuberay-operator-6b449f854b-z8gs2 MountVolume.SetUp failed for volume "cert" : secret "kuberay-webhook-server-cert" not found 93m Warning Unhealthy pod/mlflow-668464596d-g9ggc Readiness probe failed: Get "https://10.131.2.245:8443/mlflow/health": dial tcp 10.131.2.245:8443: connect: connection refused 89m Warning Unhealthy pod/mlflow-58f77fb56c-b6mts Readiness probe failed: Get "https://10.131.2.248:8443/mlflow/health": dial tcp 10.131.2.248:8443: connect: connection refused 7m53s Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 6m43s Warning FailedMount pod/spark-pi-d97elmeb1jhc7b60k7sg-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "spark-pi-d97elmeb1jhc7b60k7sg-c40ccf9f44458124-driver-podspec-conf-map" not found 6m43s Warning FailedMount pod/spark-pi-d97elmeb1jhc7b60k7sg-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-7b1b2d9f4445945a-conf-map" not found 5m24s Warning FailedMount pod/scheduled-spark-pi-d97em36b1jhc7b60k7t0-1783556906001202649-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-858e139f4446ca10-conf-map" not found 5m24s Warning FailedMount pod/scheduled-spark-pi-d97em36b1jhc7b60k7t0-1783556906001202649-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "scheduled-spark-pi-d97em36b1jhc7b60k7t0-1783556906001202649-13b0589f4446b883-driver-podspec-conf-map" not found 5m14s Warning FailedMount pod/spark-pi-c025ee9f4446eadd-exec-1 MountVolume.SetUp failed for volume "spark-conf-volume-exec" : configmap "spark-exec-2a48a49f4446f3d8-conf-map" not found 4m2s Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 4m2s Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 11s Warning FailedToRetrieveImagePullSecret pod/maas-controller-656b6867cb-vznfq Unable to retrieve some image pull secrets (maas-controller-dockercfg-tl2cp); attempting to pull the image may not succeed. 75m Warning Deleting modelregistry/default-db-registry-1783552522572 Custom Resource default-db-registry-1783552522572 is being deleted from the namespace rhoai-model-registries 156m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 76m Warning Deleting modelregistry/e2e-test-registry Custom Resource e2e-test-registry is being deleted from the namespace rhoai-model-registries 156m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 155m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 70m Warning Deleting modelregistry/test-model-registry-662874 Custom Resource test-model-registry-662874 is being deleted from the namespace rhoai-model-registries 76m Warning Deleting modelregistry/postgres-registry-1783552522572 Custom Resource postgres-registry-1783552522572 is being deleted from the namespace rhoai-model-registries 156m Warning FailedMount pod/model-registry0-849dbccdd5-zbjx8 MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-registry0-kube-rbac-proxy-config" not found 155m Warning Unhealthy pod/model-registry0-849dbccdd5-zbjx8 Readiness probe failed: Get "https://10.129.3.93:8888/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 128m Warning FailedMount pod/model-catalog-7cc8755dc9-9hkrz MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 128m Warning Unhealthy pod/model-catalog-7cc8755dc9-9hkrz Readiness probe failed: dial tcp 10.129.3.131:8080: connect: connection refused 127m Warning Unhealthy pod/model-catalog-postgres-686d85b578-5x9q5 Readiness probe errored: rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1 117m Warning FailedMount pod/model-catalog-7cc8755dc9-24s9s MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 117m Warning Unhealthy pod/model-catalog-7cc8755dc9-24s9s Readiness probe failed: dial tcp 10.131.0.37:8080: connect: connection refused 96m Warning FailedMount pod/model-catalog-7cc8755dc9-ddw9p MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 96m Warning Unhealthy pod/model-catalog-7cc8755dc9-ddw9p Readiness probe failed: dial tcp 10.128.2.32:8080: connect: connection refused 77m Warning FailedMount pod/e2e-test-registry-64df44d945-xvxx9 MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "e2e-test-registry-kube-rbac-proxy-config" not found 76m Warning Unhealthy pod/postgres-registry-1783552522572-7c64554f9f-7vgdl Readiness probe failed: Get "https://10.128.2.45:8888/healthz": dial tcp 10.128.2.45:8888: connect: connection refused 76m Warning FailedMount pod/default-db-registry-1783552522572-579f445f94-pcnxh MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "default-db-registry-1783552522572-kube-rbac-proxy-config" not found 75m Warning Unhealthy pod/default-db-registry-1783552522572-579f445f94-pcnxh Startup probe failed: Get "http://10.128.2.46:8080/readyz/isDirty": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 75m Warning Unhealthy pod/default-db-registry-1783552522572-579f445f94-pcnxh Startup probe failed: HTTP probe failed with statuscode: 503 74m Warning Unhealthy pod/model-registry-db-662874-7956bc68d9-7s9rx Readiness probe failed: mysql: [Warning] Using a password on the command line interface can be insecure.... {"level":"error","ts":"2026-07-08T23:25:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:26:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:26:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:27:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:28:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:29:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:29:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:30:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:31:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:32:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:32:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:33:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:34:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:35:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:36:06Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:36:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:37:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:38:08Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:38:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:39:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:40:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:41:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:41:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:42:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:43:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:44:06Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:44:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:45:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:46:16Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:47:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:47:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:48:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:49:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:49:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:50:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:51:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:52:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:53:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:54:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:54:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:55:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:56:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:57:19Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:58:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:58:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:59:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:00:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:00:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:01:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:02:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:03:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:04:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:04:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:05:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:06:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:07:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:08:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:09:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:10:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:10:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:11:29Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:12:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:12:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:13:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:14:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:15:18Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:15:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:16:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:17:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:18:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:18:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:19:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:20:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:20:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:21:33Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:22:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:23:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:23:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:24:20Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:25:04Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:26:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:26:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:27:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:28:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:28:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:29:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:30:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:30:59Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:31:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:32:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:33:27Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} --------------------------------------------------------------------------------------------------- ==================== STATUS REPORT ==================== ========== ODH/RHOAI STATUS REPORT (2026-07-09T00:33:58Z) ========== --- OpenShift / OCP version --- Client Version: 4.20.8 Kustomize Version: v5.6.0 Server Version: 4.19.18 Kubernetes Version: v1.32.9 --- Namespaces --- namespace/redhat-ods-applications namespace/redhat-ods-monitoring namespace/redhat-ods-operator namespace/rhoai-model-registries namespace/rhods-notebooks --- Subscriptions (redhat-ods-operator) --- NAME PACKAGE SOURCE CHANNEL rhods-operator rhods-operator rhoai-catalog-dev beta Subscription conditions (unhealthy): CatalogSourcesUnhealthy: status=False reason=AllCatalogSourcesHealthy message=all available catalogsources are healthy --- ClusterServiceVersions (redhat-ods-operator) --- NAME DISPLAY VERSION REPLACES PHASE authorino-operator.v1.4.0 Authorino Operator 1.4.0 authorino-operator.v1.3.1 Succeeded cert-manager-operator.v1.19.0 cert-manager Operator for Red Hat OpenShift 1.19.0 cert-manager-operator.v1.18.1 Succeeded custom-metrics-autoscaler.v2.19.0-1 Custom Metrics Autoscaler 2.19.0-1 custom-metrics-autoscaler.v2.18.1-2 Succeeded dns-operator.v1.4.0 DNS Operator 1.4.0 dns-operator.v1.3.1 Succeeded kueue-operator.v1.3.1 Red Hat build of Kueue 1.3.1 Succeeded limitador-operator.v1.4.0 Limitador Operator 1.4.0 limitador-operator.v1.3.1 Succeeded rhcl-operator.v1.4.0 Red Hat Connectivity Link 1.4.0 rhcl-operator.v1.3.4 Succeeded rhods-operator.3.5.0-ea.2 Red Hat OpenShift AI 3.5.0-ea.2 Succeeded servicemeshoperator3.v3.3.4 Red Hat OpenShift Service Mesh 3 3.3.4 servicemeshoperator3.v3.1.0 Succeeded --- CSVs not in Succeeded phase --- (all CSVs report phase Succeeded) --- InstallPlans (redhat-ods-operator, not Complete) --- (no non-Complete InstallPlans) --- DataScienceCluster --- NAME READY REASON default-dsc True DSC Ready summary: True reason= DSC component states: aipipelines: managementState=Managed dashboard: managementState=Managed feastoperator: managementState=Managed kserve: managementState=Managed kueue: managementState=Unmanaged llamastackoperator: managementState=n/a mlflowoperator: managementState=Managed modelregistry: managementState=Managed ogx: managementState=Managed ray: managementState=Managed sparkoperator: managementState=Managed trainer: managementState=Managed trainingoperator: managementState=Managed trustyai: managementState=Managed workbenches: managementState=Managed DSC conditions: Ready: status=True reason= message= ProvisioningSucceeded: status=True reason= message= ComponentsReady: status=True reason= message= AIPipelinesReady: status=True reason= message= DashboardReady: status=True reason= message= FeastOperatorReady: status=True reason= message= KserveLLMInferenceServiceDependencies: status=True reason= message= KserveLLMInferenceServiceWideEPDependencies: status=True reason= message= KserveReady: status=True reason= message= KueueReady: status=True reason= message= MLflowOperatorReady: status=True reason= message= MaaSPrerequisitesAvailable: status=False reason=MaaSPrerequisitesNotMet message=maas-default-gateway is missing required annotations: security.opendatahub.io/authorino-tls-bootstrap="true" ModelControllerReady: status=True reason= message= ModelRegistryReady: status=True reason= message= ModelsAsServiceReady: status=True reason=Reconciled message=MaaS platform manifests applied and maas-api deployment is available OGXReady: status=True reason= message= RayReady: status=True reason= message= SparkOperatorReady: status=True reason= message= TrainerReady: status=True reason= message= TrainingOperatorReady: status=True reason= message= TrustyAIReady: status=True reason= message= WorkbenchesReady: status=True reason= message= --- DSCInitialization --- NAME AGE PHASE CREATED AT default-dsci 9d Ready 2026-06-29T13:43:18Z DSCInitialization conditions: ReconcileComplete: status=True reason=ReconcileCompleted message=Reconcile completed successfully Available: status=True reason=ReconcileCompleted message=Reconcile completed successfully Progressing: status=False reason=ReconcileCompleted message=Reconcile completed successfully Degraded: status=False reason=ReconcileCompleted message=Reconcile completed successfully Upgradeable: status=True reason=ReconcileCompleted message=Reconcile completed successfully MonitoringReady: status=True reason=Ready message=Monitoring stack is initialized Ready: status=True reason= message= ProvisioningSucceeded: status=True reason= message= MonitoringStackAvailable: status=False reason=MetricsNotConfigured message=Metrics not configured in DSCI CR TempoAvailable: status=False reason=TracesNotConfigured message=Traces not configured in DSCI CR OpenTelemetryCollectorAvailable: status=False reason=MetricsNotConfiguredAndTracesNotConfigured message=Metrics not configured in DSCI CR Traces not configured in DSCI CR AlertingAvailable: status=False reason=AlertingNotConfigured message=Alerting not configured in DSCI CR ThanosQuerierAvailable: status=False reason=MetricsNotConfigured message=Metrics not configured in DSCI CR PersesAvailable: status=False reason=MetricsNotConfiguredAndTracesNotConfigured message=Perses requires at least Metrics or Traces to be configured NodeMetricsEndpointAvailable: status=False reason=MetricsNotConfigured message=Metrics not configured in DSCI CR --- Component CRs (Ready) --- NAME READY REASON URL dashboard.components.platform.opendatahub.io/default-dashboard True NAME READY REASON datasciencepipelines.components.platform.opendatahub.io/default-datasciencepipelines True NAME READY REASON feastoperator.components.platform.opendatahub.io/default-feastoperator True NAME READY REASON kserve.components.platform.opendatahub.io/default-kserve True NAME READY REASON kueue.components.platform.opendatahub.io/default-kueue True NAME READY REASON mlflowoperator.components.platform.opendatahub.io/default-mlflowoperator True NAME READY REASON URI modelcontroller.components.platform.opendatahub.io/default-modelcontroller True NAME READY REASON modelregistry.components.platform.opendatahub.io/default-modelregistry True NAME READY REASON modelsasservice.components.platform.opendatahub.io/default-modelsasservice True NAME READY REASON ogx.components.platform.opendatahub.io/default-ogx True NAME READY REASON ray.components.platform.opendatahub.io/default-ray True NAME READY REASON sparkoperator.components.platform.opendatahub.io/default-sparkoperator True NAME READY REASON trainer.components.platform.opendatahub.io/default-trainer True NAME READY REASON trainingoperator.components.platform.opendatahub.io/default-trainingoperator True NAME READY REASON trustyai.components.platform.opendatahub.io/default-trustyai True NAME READY REASON workbenches.components.platform.opendatahub.io/default-workbenches True --- Deployments not fully ready (ODH/RHOAI namespaces) --- (all deployments report ready replicas == desired) --- Pods not Running/Succeeded (ODH/RHOAI namespaces) --- (all pods Running or Succeeded) --------------------------------------------------------------------------------------------------- ==================== EVENTS ==================== ========== ODH/RHOAI EVENTS (Warning / Error / Failed) ========== --- namespace/redhat-ods-applications --- LAST SEEN TYPE REASON OBJECT MESSAGE 132m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 129m Warning Unhealthy pod/kserve-controller-manager-6c8cb86d7f-st4s8 Readiness probe failed: Get "http://10.129.3.45:8081/readyz": dial tcp 10.129.3.45:8081: connect: connection refused 129m Warning FailedToRetrieveImagePullSecret pod/selector-migration-n4q8g Unable to retrieve some image pull secrets (feast-operator-controller-manager-dockercfg-xvxzx); attempting to pull the image may not succeed. 129m Warning Unhealthy pod/model-serving-api-78bbbb9bf8-zmqbh Readiness probe failed: Get "https://10.130.3.13:8443/readyz": context deadline exceeded 129m Warning FailedCreate replicaset/feast-operator-controller-manager-744655f689 Error creating: pods "feast-operator-controller-manager-744655f689-" is forbidden: error looking up service account redhat-ods-applications/feast-operator-controller-manager: serviceaccount "feast-operator-controller-manager" not found 129m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-n6hks MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 129m Warning FailedCreate job/selector-migration Error creating: pods "selector-migration-" is forbidden: error looking up service account redhat-ods-applications/feast-operator-controller-manager: serviceaccount "feast-operator-controller-manager" not found 129m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-mvvkb MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 129m Warning FailedMount pod/odh-model-controller-6b647f87c-g2lwp MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 129m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 129m Warning FailedMount pod/odh-notebook-controller-manager-7f6b58f576-dkjsh MountVolume.SetUp failed for volume "webhook-cert" : secret "odh-notebook-controller-webhook-cert" not found 129m Warning FailedCreate replicaset/notebook-controller-deployment-5d7d79cf5b Error creating: pods "notebook-controller-deployment-5d7d79cf5b-" is forbidden: error looking up service account redhat-ods-applications/notebook-controller-service-account: serviceaccount "notebook-controller-service-account" not found 128m Warning FailedToUpdateEndpointSlices service/ogx-k8s-operator-controller-manager-metrics-service Error updating Endpoint Slices for Service redhat-ods-applications/ogx-k8s-operator-controller-manager-metrics-service: failed to delete ogx-k8s-operator-controller-manager-metrics-service-8sngc EndpointSlice for Service redhat-ods-applications/ogx-k8s-operator-controller-manager-metrics-service: endpointslices.discovery.k8s.io "ogx-k8s-operator-controller-manager-metrics-service-8sngc" not found 128m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 128m Warning CalculateExpectedPodCountFailed poddisruptionbudget/ogx-k8s-operator-controller-manager-pdb Failed to calculate the number of expected pods: found no controllers for pod "ogx-k8s-operator-controller-manager-67866ffc79-n6hks" 128m Warning CalculateExpectedPodCountFailed poddisruptionbudget/ogx-k8s-operator-controller-manager-pdb Failed to calculate the number of expected pods: found no controllers for pod "ogx-k8s-operator-controller-manager-67866ffc79-pdzlm" 127m Warning FailedMount pod/data-science-pipelines-operator-controller-manager-7fc7f762ncnr MountVolume.SetUp failed for volume "config" : configmap "data-science-pipelines-operator-dspo-config" not found 127m Warning FailedCreate replicaset/rhods-dashboard-6b5cc9785c Error creating: pods "rhods-dashboard-6b5cc9785c-" is forbidden: error looking up service account redhat-ods-applications/rhods-dashboard: serviceaccount "rhods-dashboard" not found 127m Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 127m Warning FailedMount pod/kuberay-operator-6b449f854b-hzkwl MountVolume.SetUp failed for volume "cert" : secret "kuberay-webhook-server-cert" not found 126m Warning FailedMount pod/spark-pi-d97ctlmb1jhc7b532630-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-85b93b9f43d82162-conf-map" not found 126m Warning FailedMount pod/spark-pi-d97ctlmb1jhc7b532630-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "spark-pi-d97ctlmb1jhc7b532630-fd5abb9f43d81031-driver-podspec-conf-map" not found 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-26lrs Error: ErrImagePull 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-26lrs Failed to pull image "registry.redhat.io/rhoai/odh-mod-arch-model-registry-rhel9@sha256:1ce99d94d4af0a582e3a98a4e6a7758652a98c2e4dd6fff2380cfdd925a6f5cf": copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/a9/a9d40ceb865f72b9ecca3cc5dca6cac11a6e344d4273c7d2f67ca4d4e1556d9c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260708%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260708T222848Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=0b371b64ab9d8da3792e9bf06f65e6cc14c3115729524771caa6005a42b09dba®ion=us-east-1&namespace=rhoai&username=rhoai+rhoai_devops_konfux_bot&repo_name=odh-mod-arch-model-registry-rhel9&akamai_signature=exp=1783550628~hmac=14eac819a45da307f16ac74985ff394b85b4f99238add35156a120dc9a909fc9": EOF 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-26lrs Error: ImagePullBackOff 124m Warning FailedMount pod/scheduled-spark-pi-d97cu26b1jhc7b532640-1783549734000822948-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-1d7a259f43d95bcf-conf-map" not found 124m Warning FailedMount pod/scheduled-spark-pi-d97cu26b1jhc7b532640-1783549734000822948-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "scheduled-spark-pi-d97cu26b1jhc7b532640-1783549734000822948-f707b19f43d949cd-driver-podspec-conf-map" not found 118m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-86x4p MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 117m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 117m Warning InvalidConfiguration servicemonitor/workload-variant-autoscaler-controller-manager-metrics-monitor ServiceMonitor workload-variant-autoscaler-controller-manager-metrics-monitor was rejected due to invalid configuration: failed to get authorization token of type "": unable to get secret "workload-variant-autoscaler-metrics-reader-token": secrets "workload-variant-autoscaler-metrics-reader-token" not found 114m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 111m Warning FailedMount pod/odh-model-controller-54bf759f64-gt5l2 MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 111m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 110m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 110m Warning FailedCreate replicaset/trustyai-service-operator-controller-manager-6f7f88c874 Error creating: pods "trustyai-service-operator-controller-manager-6f7f88c874-" is forbidden: error looking up service account redhat-ods-applications/trustyai-service-operator-controller-manager: serviceaccount "trustyai-service-operator-controller-manager" not found 110m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 110m Warning FailedToUpdateEndpoint endpoints/mlflow-operator-controller-manager-metrics-service Failed to update endpoint redhat-ods-applications/mlflow-operator-controller-manager-metrics-service: Operation cannot be fulfilled on endpoints "mlflow-operator-controller-manager-metrics-service": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/redhat-ods-applications/mlflow-operator-controller-manager-metrics-service, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 330380f2-ca5a-4f73-9613-db635448e8e7, UID in object meta: 108m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 107m Warning FailedMount pod/odh-model-controller-54bf759f64-2n87k MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 107m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 107m Warning FailedMount pod/kserve-controller-manager-6c8cb86d7f-nrmd4 MountVolume.SetUp failed for volume "cert" : secret "kserve-webhook-server-cert" not found 107m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 105m Warning BackOff pod/maas-controller-656b6867cb-vznfq Back-off restarting failed container manager in pod maas-controller-656b6867cb-vznfq_redhat-ods-applications(17cbcc19-188b-4f00-bbeb-825e0a39a8f1) 100m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 99m Warning FailedMount pod/odh-model-controller-54bf759f64-wgh9l MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 99m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 99m Warning FailedMount pod/model-serving-api-78bbbb9bf8-r7twk MountVolume.SetUp failed for volume "tls-certs" : secret "model-serving-api-tls" not found 99m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 98m Warning FailedMount pod/kserve-controller-manager-6c8cb86d7f-6b97l MountVolume.SetUp failed for volume "cert" : secret "kserve-webhook-server-cert" not found 98m Warning FailedMount pod/odh-model-controller-54bf759f64-bt4fr MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 97m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 97m Warning FailedMount pod/kserve-controller-manager-6c8cb86d7f-rj9zj MountVolume.SetUp failed for volume "cert" : secret "kserve-webhook-server-cert" not found 96m Warning FailedCreate replicaset/trustyai-service-operator-controller-manager-6f7f88c874 Error creating: pods "trustyai-service-operator-controller-manager-6f7f88c874-" is forbidden: error looking up service account redhat-ods-applications/trustyai-service-operator-controller-manager: serviceaccount "trustyai-service-operator-controller-manager" not found 96m Warning FailedMount pod/mlflow-operator-controller-manager-8646f94cf4-4jxqg MountVolume.SetUp failed for volume "metrics-cert" : secret "controller-manager-metrics-tls" not found 96m Warning FailedMount pod/data-science-pipelines-operator-controller-manager-7fc7f76glsfg MountVolume.SetUp failed for volume "config" : configmap "data-science-pipelines-operator-dspo-config" not found 96m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-cq44q MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 96m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-j9r4m MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 96m Warning FailedMount pod/odh-notebook-controller-manager-7f6b58f576-dfxz4 MountVolume.SetUp failed for volume "webhook-cert" : secret "odh-notebook-controller-webhook-cert" not found 96m Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 96m Warning FailedCreate replicaset/notebook-controller-deployment-5d7d79cf5b Error creating: pods "notebook-controller-deployment-5d7d79cf5b-" is forbidden: error looking up service account redhat-ods-applications/notebook-controller-service-account: serviceaccount "notebook-controller-service-account" not found 96m Warning BackOff pod/trustyai-service-operator-controller-manager-6f7f88c874-242cm Back-off restarting failed container manager in pod trustyai-service-operator-controller-manager-6f7f88c874-242cm_redhat-ods-applications(a5408679-a4ef-468f-9f5a-1f7c755f16d0) 96m Warning FailedCreate replicaset/rhods-dashboard-6b5cc9785c Error creating: pods "rhods-dashboard-6b5cc9785c-" is forbidden: error looking up service account redhat-ods-applications/rhods-dashboard: serviceaccount "rhods-dashboard" not found 96m Warning FailedMount pod/kuberay-operator-6b449f854b-z8gs2 MountVolume.SetUp failed for volume "cert" : secret "kuberay-webhook-server-cert" not found 93m Warning Unhealthy pod/mlflow-668464596d-g9ggc Readiness probe failed: Get "https://10.131.2.245:8443/mlflow/health": dial tcp 10.131.2.245:8443: connect: connection refused 89m Warning Unhealthy pod/mlflow-58f77fb56c-b6mts Readiness probe failed: Get "https://10.131.2.248:8443/mlflow/health": dial tcp 10.131.2.248:8443: connect: connection refused 7m53s Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 6m43s Warning FailedMount pod/spark-pi-d97elmeb1jhc7b60k7sg-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "spark-pi-d97elmeb1jhc7b60k7sg-c40ccf9f44458124-driver-podspec-conf-map" not found 6m43s Warning FailedMount pod/spark-pi-d97elmeb1jhc7b60k7sg-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-7b1b2d9f4445945a-conf-map" not found 5m24s Warning FailedMount pod/scheduled-spark-pi-d97em36b1jhc7b60k7t0-1783556906001202649-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-858e139f4446ca10-conf-map" not found 5m24s Warning FailedMount pod/scheduled-spark-pi-d97em36b1jhc7b60k7t0-1783556906001202649-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "scheduled-spark-pi-d97em36b1jhc7b60k7t0-1783556906001202649-13b0589f4446b883-driver-podspec-conf-map" not found 5m14s Warning FailedMount pod/spark-pi-c025ee9f4446eadd-exec-1 MountVolume.SetUp failed for volume "spark-conf-volume-exec" : configmap "spark-exec-2a48a49f4446f3d8-conf-map" not found 4m2s Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 4m2s Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 11s Warning FailedToRetrieveImagePullSecret pod/maas-controller-656b6867cb-vznfq Unable to retrieve some image pull secrets (maas-controller-dockercfg-tl2cp); attempting to pull the image may not succeed. --- namespace/redhat-ods-monitoring --- (no events) --- namespace/redhat-ods-operator --- (no events) --- namespace/rhoai-model-registries --- LAST SEEN TYPE REASON OBJECT MESSAGE 75m Warning Deleting modelregistry/default-db-registry-1783552522572 Custom Resource default-db-registry-1783552522572 is being deleted from the namespace rhoai-model-registries 156m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 76m Warning Deleting modelregistry/e2e-test-registry Custom Resource e2e-test-registry is being deleted from the namespace rhoai-model-registries 156m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 155m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 70m Warning Deleting modelregistry/test-model-registry-662874 Custom Resource test-model-registry-662874 is being deleted from the namespace rhoai-model-registries 76m Warning Deleting modelregistry/postgres-registry-1783552522572 Custom Resource postgres-registry-1783552522572 is being deleted from the namespace rhoai-model-registries 156m Warning FailedMount pod/model-registry0-849dbccdd5-zbjx8 MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-registry0-kube-rbac-proxy-config" not found 155m Warning Unhealthy pod/model-registry0-849dbccdd5-zbjx8 Readiness probe failed: Get "https://10.129.3.93:8888/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 128m Warning FailedMount pod/model-catalog-7cc8755dc9-9hkrz MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 128m Warning Unhealthy pod/model-catalog-7cc8755dc9-9hkrz Readiness probe failed: dial tcp 10.129.3.131:8080: connect: connection refused 127m Warning Unhealthy pod/model-catalog-postgres-686d85b578-5x9q5 Readiness probe errored: rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1 117m Warning FailedMount pod/model-catalog-7cc8755dc9-24s9s MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 117m Warning Unhealthy pod/model-catalog-7cc8755dc9-24s9s Readiness probe failed: dial tcp 10.131.0.37:8080: connect: connection refused 96m Warning FailedMount pod/model-catalog-7cc8755dc9-ddw9p MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 96m Warning Unhealthy pod/model-catalog-7cc8755dc9-ddw9p Readiness probe failed: dial tcp 10.128.2.32:8080: connect: connection refused 77m Warning FailedMount pod/e2e-test-registry-64df44d945-xvxx9 MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "e2e-test-registry-kube-rbac-proxy-config" not found 76m Warning Unhealthy pod/postgres-registry-1783552522572-7c64554f9f-7vgdl Readiness probe failed: Get "https://10.128.2.45:8888/healthz": dial tcp 10.128.2.45:8888: connect: connection refused 76m Warning FailedMount pod/default-db-registry-1783552522572-579f445f94-pcnxh MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "default-db-registry-1783552522572-kube-rbac-proxy-config" not found 75m Warning Unhealthy pod/default-db-registry-1783552522572-579f445f94-pcnxh Startup probe failed: Get "http://10.128.2.46:8080/readyz/isDirty": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 75m Warning Unhealthy pod/default-db-registry-1783552522572-579f445f94-pcnxh Startup probe failed: HTTP probe failed with statuscode: 503 74m Warning Unhealthy pod/model-registry-db-662874-7956bc68d9-7s9rx Readiness probe failed: mysql: [Warning] Using a password on the command line interface can be insecure.... --- namespace/rhods-notebooks --- LAST SEEN TYPE REASON OBJECT MESSAGE --------------------------------------------------------------------------------------------------- ==================== OPERATOR LOG HIGHLIGHTS ==================== ========== ODH/RHOAI LOG HIGHLIGHTS (errors, webhooks, reconcile failures) ========== Source: rhods-operator pods in redhat-ods-operator (since-time=2026-07-08T21:22:34Z) --- redhat-ods-operator/rhods-operator-6f6bb89fdf-59gvk --- {"level":"error","ts":"2026-07-08T23:25:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:26:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:26:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:27:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:28:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:29:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:29:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:30:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:31:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:32:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:32:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:33:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:34:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:35:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:36:06Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:36:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:37:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:38:08Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:38:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:39:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:40:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:41:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:41:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:42:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:43:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:44:06Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:44:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:45:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:46:16Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:47:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:47:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:48:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:49:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:49:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:50:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:51:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:52:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:53:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:54:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:54:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:55:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:56:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:57:19Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:58:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:58:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:59:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:00:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:00:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:01:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:02:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:03:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:04:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:04:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:05:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:06:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:07:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:08:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:09:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:10:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:10:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:11:29Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:12:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:12:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:13:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:14:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:15:18Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:15:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:16:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:17:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:18:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:18:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:19:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:20:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:20:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:21:33Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:22:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:23:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:23:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:24:20Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:25:04Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:26:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:26:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:27:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:28:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:28:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:29:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:30:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:30:59Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:31:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:32:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:33:27Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} --------------------------------------------------------------------------------------------------- ==================== OPERATOR POD LOG: rhods-operator-6f6bb89fdf-59gvk.log ==================== {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:24:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:24:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea1a8bec-3ee5-4df1-9f23-40d9c07d471d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:24:51Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"maas-subs-project-274868","reconcileID":"2057f654-f3d5-465f-92c5-bd209904d458"} {"level":"info","ts":"2026-07-08T23:24:51Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"maas-subs-project-274868","reconcileID":"0b386a22-7597-48ab-8469-e5e126a28359"} {"level":"info","ts":"2026-07-08T23:24:51Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"maas-subs-project-274868","reconcileID":"63a0ad03-b587-4775-be78-2ccda60001bd"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:25:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18500cb3-04d9-4c72-8714-b244b40bcf86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:25:18Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2622b245-add3-48a7-b388-3b65b6397a2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:25:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:25:33Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:25:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"39a815f3-941c-4f32-9962-8553bfea744a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:26:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"747637db-a653-459a-8830-5d3b127f051b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:26:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:26:18Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88bb3de4-9360-4895-b896-37e6eeac19fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:27Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"cypress-test-delete-applied-hp-105453","reconcileID":"bfd1aa38-3939-421b-a003-0c8081672a51"} {"level":"info","ts":"2026-07-08T23:26:27Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"cypress-test-delete-applied-hp-105453","reconcileID":"2766ee30-5081-4d4c-9626-2327b625c872"} {"level":"info","ts":"2026-07-08T23:26:27Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"cypress-test-delete-applied-hp-105453","reconcileID":"dc316cc2-3fcf-4f40-8469-06e11cc018c6"} {"level":"info","ts":"2026-07-08T23:26:28Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8"} {"level":"info","ts":"2026-07-08T23:26:28Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8"} {"level":"info","ts":"2026-07-08T23:26:28Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-08T23:26:28Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-08T23:26:28Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:28Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:28Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-08T23:26:29Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-08T23:26:29Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-08T23:26:29Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ca522726-2f55-45f0-852e-6fe5a87c3ba8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-08T23:26:30Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"840d5ca6-c76c-491c-8f39-9efe97f56385","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-08T23:26:30Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-08T23:26:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b24e252c-6b2a-4457-9190-721a1c8a342e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-08T23:26:31Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"93b38bbc-ea48-4241-acd7-f79c07701e57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:26:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"016354c4-f419-4902-977c-6157a59ca92a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:26:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"44bbd5f6-587b-465b-9102-ce34a22d5710","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:26:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2155b006-f046-409c-bb54-ae447e6d1e85","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ef80e9d1-53c7-40da-aba5-c8bddcd55728","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:37Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"6a12bea1-b296-4e48-919d-3f9a8f877974","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"6a12bea1-b296-4e48-919d-3f9a8f877974","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9422ca10-cc5b-44ca-adbb-0b16f89303b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:26:37Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"8f61fa04-8ae3-4de0-9629-bfa96f849ea6","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:26:37Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"8f61fa04-8ae3-4de0-9629-bfa96f849ea6","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:26:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:26:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0fd7c4dd-2b0d-4268-87be-1aa240e2d5e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:26:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fec6defb-a9cc-49ef-9a45-69101c2aee40","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:18Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8a04539f-e71b-43d1-840a-bca2f2fa083e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f989081-c977-490d-931b-f60b5d9a66ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d90143b2-b748-485a-af00-9135191d5ed8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:26Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ea5a2444-6943-4334-8cc6-5377123e4040","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:27:26Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ea5a2444-6943-4334-8cc6-5377123e4040","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:28Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13355a39-682c-41f2-9a0e-c51f5adec59c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:28Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"892ef98e-8518-4878-bf02-d1740ad816a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:28Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0bb20349-a97f-41f8-badc-519337728e98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f064a8d-ff0f-4eb6-a0e0-8a9206b6b9db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:28Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"5e1dfd36-1597-4c5f-95df-ef86ff2cfac2","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:27:28Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"5e1dfd36-1597-4c5f-95df-ef86ff2cfac2","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3fd08fe0-21cd-4c56-a2ee-353e7f76e450","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"1ebc148b-2165-4eb8-95b5-c6fdce06c83a","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"1ebc148b-2165-4eb8-95b5-c6fdce06c83a","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5ac5da0-5023-47a0-bfb2-bce24544446e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:33Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c3d5db5d-03d4-49bc-8b3a-18b6eeb57eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:27:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:27:36Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"522b3865-ada8-4e3d-85ca-ab644db68b40"} {"level":"info","ts":"2026-07-08T23:27:36Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"9303ca0c-8d38-4b19-be20-ab1ca270ac7e"} {"level":"info","ts":"2026-07-08T23:27:36Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"672adcc4-c38a-4ad9-9579-7e413f2c0ea0"} {"level":"info","ts":"2026-07-08T23:27:46Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a24c272a-8021-4ec5-96ab-fdf70c669610","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d792d50-18f0-48fd-849e-9d8f269c8bd9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d7bb0695-a1dc-44fc-88e8-447ad5f0ee48","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:27:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3f5b46f7-7e4f-4867-ad78-5666da10099a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"baf8e035-5f72-4d4f-bdd6-4a53d177fe14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:49Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"6d9d7450-8024-4efd-bb3a-61ecfd0d2e79","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"6d9d7450-8024-4efd-bb3a-61ecfd0d2e79","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:27:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2fc2b83f-fb1f-49da-92a5-58872d72f6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:27:51Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"137a5f26-e665-4ed0-bbb1-57ae6cc4d7f0","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:27:51Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"137a5f26-e665-4ed0-bbb1-57ae6cc4d7f0","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-08T23:28:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:28:18Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e36ef5-86a8-46e2-a600-119cf94432c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:28:33Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"63f43fe2-d39f-41d8-8e4b-a04d2c2d70a0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:28:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7211e473-ba1a-4010-8153-749ec57de51e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:28:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2b1e118-4eb9-4880-a455-cd1edf7ffdad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:28:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53e65610-a45e-498d-bbcf-c9c3f708663e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e0e43732-48b3-4b12-9609-4d1ed3fde83c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ea7d8636-0284-4536-be72-ac5c6bc61f09","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ea7d8636-0284-4536-be72-ac5c6bc61f09","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0755a33e-9250-4050-b9a7-49cbda54145e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4b86e582-45c5-48c1-9c01-2ae61c328605","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6eb5fa17-22a4-4b2f-afd1-6de9a0e2a3d1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"60517614-dc63-445d-8564-0f8cf2e478a5","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"60517614-dc63-445d-8564-0f8cf2e478a5","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d90abc-522e-44d3-ac44-f2b3c60a39c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:28:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:28:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0abcf053-08eb-4094-9e84-3edd4f54a2b7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:29:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:29:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:29:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b335282-5513-45ca-9a8b-473df38820ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:29:42Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-08T23:29:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:29:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:29:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca716441-cbd3-49bc-a149-1eaacfc3d916","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:30:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d53a8e9b-8e89-4172-b97a-22e048807f4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:30:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:30:33Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faa8d068-62e0-41c7-a0aa-633b508ff708","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:30:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:30:48Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"963841dd-d4f6-4028-a846-23fbfeedb343","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:31:18Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:31:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"169f2a6e-c8f5-42d0-a012-b7e954733f93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:31:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:31:33Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:31:33Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a7223af-cc5c-4762-aafb-7522cbb5b60d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:31:51Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-08T23:32:07Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"75009fb3-3fdc-47ff-ada8-c48588f97808"} {"level":"info","ts":"2026-07-08T23:32:07Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"53cca3a7-56f6-43ac-b862-8c0ad3adb0f1"} {"level":"info","ts":"2026-07-08T23:32:07Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"a6581a54-9481-4e45-a7c6-c50ed41100d4"} {"level":"error","ts":"2026-07-08T23:32:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:32:19Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"65818a2e-0049-4478-9dd9-147de7e9fd02","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:32:34Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"08a4000a-6063-49d8-97f6-5492587fb408","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:32:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:32:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:32:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dc612bc0-907c-4df8-b5c6-470d45a72f8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:33:04Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:33:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d2fdc7bb-b052-4ede-ae01-298f9cb1fa58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:33:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:34:04Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"77b94ebc-0a30-4b58-9e33-7498c1ba8418","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:34:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:34:19Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"359d2410-3205-4ba5-9208-faa3c4d9f179","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:34:39Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"642eca42-0620-42c1-889d-2bf4c4362eed"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"642eca42-0620-42c1-889d-2bf4c4362eed"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"642eca42-0620-42c1-889d-2bf4c4362eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"642eca42-0620-42c1-889d-2bf4c4362eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"642eca42-0620-42c1-889d-2bf4c4362eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"642eca42-0620-42c1-889d-2bf4c4362eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"642eca42-0620-42c1-889d-2bf4c4362eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"642eca42-0620-42c1-889d-2bf4c4362eed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-08T23:34:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:34:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"39b8b538-3bc6-4124-9e53-4c44255e4f17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:35:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:35:19Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5d39a3a7-e00d-48fe-a522-22be07867032","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:35:19Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:35:34Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a1be3af6-be0d-4fd6-a9a7-76f51d7fce65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:35:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:35:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48d6b4d7-66bd-45d6-8f72-e5a1fc40f2a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:36:06Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:36:19Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:36:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5694fd2b-1915-468b-96ba-ca5ac14ad2be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:36:34Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:36:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7d6fdcc5-db9c-4b1f-bb04-f32f0098ab15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:36:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:37:34Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:37:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"46e2d610-a14d-437e-89dc-f60299ae2da4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:37:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:38:08Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:38:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:39:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:40:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:40:34Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:34Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5"} {"level":"info","ts":"2026-07-08T23:40:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e6f2e4d2-045c-4d60-882d-fa9e9ff134b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:40:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59bfc73e-d52c-4622-8ee1-c32d371fc719","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:40:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"224ebe2b-29ac-43a3-92cd-c584ec7b05d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:40:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"179ddd6b-2b1b-4708-ba90-ed8c898eb933","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a5bfea5e-9a72-4d14-a9d0-48d045da6cc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"22e4ac62-cb01-4893-ae3e-7b250e4250b6","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"22e4ac62-cb01-4893-ae3e-7b250e4250b6","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ff4c7b7-9a92-4ab2-b3c3-ad220abad45c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:35Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f7f324a4-dd37-4dae-b5b0-ab63b56281d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0ee342a0-593d-449c-9bbb-54b851419a75","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"83d9cdf4-7017-44be-9a86-4c1244a57692","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"83d9cdf4-7017-44be-9a86-4c1244a57692","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6150d3b4-63a5-49b2-8e2d-9b29dbb90eda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:40:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bdf9698d-2891-4b05-9416-e1a1b8162a93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:40:55Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"c1260811-790e-4065-8121-b2d767519332"} {"level":"info","ts":"2026-07-08T23:40:55Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"9075a2fc-baa2-438a-a619-d764b4a86dfb"} {"level":"info","ts":"2026-07-08T23:40:55Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"f49c5ab3-dece-412f-be42-359a13559849"} {"level":"error","ts":"2026-07-08T23:41:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:41:04Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:04Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d4524c45-732e-4caa-9b35-dd016287a944","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:41:19Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:19Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"79ce9865-f9a7-4cc0-988f-4fd8f0f29b8d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:41:20Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:41:34Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:41:34Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ef2fcd34-fa8a-49c5-8628-32e7050a843c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:41:37Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-08T23:41:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:42:05Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:42:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"56ed35dd-ab4e-4c55-813f-90a9c2f36ada","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:42:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:42:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6c3a2c0c-e8e2-4925-a088-7e5b58e38c8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:42:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:42:46Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-08T23:43:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:43:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:43:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c7f9ec84-144f-4289-80f4-c674a7877a5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:44:05Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:44:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e23ec599-395b-42ae-ac29-22843249cbba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:44:06Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:44:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:45:20Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:45:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bdd95886-a832-4a4b-b9cd-48fb21173bda","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:45:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:45:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"894477a0-3b8d-4b9a-a6e0-b53a39dfe660","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:45:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:46:16Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:46:20Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:46:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"35711b1e-ee20-4072-a0bc-ff94cc3ec4c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:46:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:46:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"efb6d867-3faa-4a65-8bb0-0f64ce42f385","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:47:05Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:47:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae707f72-7e26-4d07-b16e-aa47f5b9cbed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:47:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:47:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:48:20Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:48:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a613008-15fa-4c7f-a466-d395b46d3b4b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:48:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:48:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:48:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87456491-252c-4bbc-8bc3-8f35ed6c8ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:49:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:49:12Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"568ab165-3a58-41ae-862c-14cbee8aea85"} {"level":"info","ts":"2026-07-08T23:49:12Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"87b967fe-78e5-40b6-98ea-994ec74ef7de"} {"level":"info","ts":"2026-07-08T23:49:12Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"6635c491-8058-4eac-b0b5-e780f5b20ef2"} {"level":"info","ts":"2026-07-08T23:49:21Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:49:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:49:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b549af0c-6d6d-40e7-a9dc-b6891115d03e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:49:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:49:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:49:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8144c1e0-bfe5-48f9-ba83-a9e7e8821157","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:49:53Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:50:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a618998f-a66c-42b5-b2fb-275c90b548a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:50:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:35Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9fbf00-36d1-4e63-a70a-3e3c71e6ac85","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"48dd8adf-c90b-4c70-8528-ab3cb9ede7f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"50153447-9230-402a-a0e6-7accc403834f","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"50153447-9230-402a-a0e6-7accc403834f","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3d559d0a-9448-49df-8646-ef20648cc3ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c0f39448-19e5-456c-a3c8-329321bbc5bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f4ec9bd9-edc2-450d-9160-e8b3a1f1bde0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"13d9dcfd-2202-4865-a4bc-42e10068063f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"aee7b997-5be5-4411-9dde-8fa4ebfb5385","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"aee7b997-5be5-4411-9dde-8fa4ebfb5385","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ce0c7a9d-110e-4a86-be53-d3b1b8df26ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7bc4d9c1-99c8-4b3a-9cf5-c9c7d69d72e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"af2df4b1-5ed2-447d-a755-8060e9f79421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:50:38Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e2b32d1e-b64c-4f11-9682-3db7b1de10e7","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:50:38Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e2b32d1e-b64c-4f11-9682-3db7b1de10e7","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-08T23:50:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:50:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:50:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f1b6f30-e2ec-4fae-944d-857f66f07967","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:51:05Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58bda7e5-b681-49e8-bb64-ed0cf0ceb50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:51:20Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:20Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4468fe3e-ab10-4c14-a702-fe8a7608e50b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:51:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:51:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:51:50Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d77755fb-1a29-4974-a517-56a45be5aab1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:52:05Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"59227953-c80a-4c5a-8ccf-671caec108bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:52:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e4e870ef-d2fa-4137-8108-3259a72bafec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:52:37Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-08T23:52:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:52:51Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:52:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1704fc6d-c3df-45e4-9a69-06b5f8593de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:53:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:54:06Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bba2902a-bb4c-4a45-bb58-a5c2e8c1464d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:54:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:54:21Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febafac6-48c8-489d-b2ae-7b7c4a11599e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:54:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:54:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fdfde4a-1512-421a-8c6d-1aa576060194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:54:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:55:09Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:55:21Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:55:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8354b17e-8dd2-4f9d-92f3-59b93e0b6fb0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:55:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:55:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"297cd37e-e54b-4f49-a7fa-42f8660212b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:55:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"35df3786-899d-40cb-8be8-538639b45ff5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:56:24Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"cb57dc28-521b-4050-8835-755409ed9b77","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-08T23:56:24Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"cb57dc28-521b-4050-8835-755409ed9b77","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-08T23:56:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:56:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1c5c383e-3b13-4e17-bc8f-98ad82641590","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:56:51Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:56:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"980462c7-616a-4341-afc9-e2dd344bb540","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:57:06Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:57:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa823ddc-f2a7-4218-b128-6f91353e2983","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:57:19Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:58:01Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"9284305d-e7d6-4960-b2b8-9e15cb5eb916"} {"level":"info","ts":"2026-07-08T23:58:01Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"0aa70641-1228-4a41-842f-e82bdbaffe2a"} {"level":"info","ts":"2026-07-08T23:58:01Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"820eca8b-bbe5-438a-8871-22c9120e0683"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:58:06Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"619032ef-1f96-4e72-a30a-b09e2796db9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:58:12Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-08T23:58:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:58:21Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7757de79-06a5-4f4d-aefe-fa460b194768","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:58:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dd4aafdb-9a84-45c2-adaa-a9669a2b03c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-08T23:58:51Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-08T23:58:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1ab25efe-36ea-43cb-ae33-49e652a373f7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-08T23:58:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-08T23:59:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-08T23:59:59Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-09T00:00:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:00:13Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24"} {"level":"info","ts":"2026-07-09T00:00:13Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24"} {"level":"info","ts":"2026-07-09T00:00:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:00:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:00:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:00:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:00:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:00:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:00:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"aea45834-2373-4941-8c81-e232863c2a24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:00:21Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:21Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8922bfb-da46-4d1c-bf61-35bafe24cd59","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:00:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f"} {"level":"info","ts":"2026-07-09T00:00:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8d5d8182-f354-48b3-9eae-4053c1e8579f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:00:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6b280808-31d5-474d-9ec5-5532f80a43b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:00:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"006d8bf5-70eb-4bbc-a50a-948abbe35ab0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:00:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"410ea765-bcce-4426-9c43-38c9adc3c88c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9aff42c-1208-4e40-9bf3-ffaaf3c7184e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"367afd9a-798d-44be-9741-4f3855bf1bbd","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"367afd9a-798d-44be-9741-4f3855bf1bbd","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a21a9764-4778-4e1e-99b7-a02a3bae71ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"761759dd-47d1-4799-a387-c05954b6fc98","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"55303e14-7c98-42c9-b655-bda46d605de2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2e14ce65-c2d6-4290-9780-ea2cee660816","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2e14ce65-c2d6-4290-9780-ea2cee660816","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30b3f82f-8431-4a77-bc46-2599cd0fb46f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"94101108-207f-4884-819b-9a1b51388de6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5a407cc0-589f-4dc5-adb3-b65cc8e49f58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"cb7824a1-c3a2-4a22-b8d3-00ea5576723b","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"cb7824a1-c3a2-4a22-b8d3-00ea5576723b","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb1d2b56-51c4-4d4c-aad7-e4d903e6dd9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f2fc76d6-1850-43b4-a1a8-58e4dd16abe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"58e95677-9b09-440f-bfa7-0873ced69378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:00:40Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0d4162ab-aa3e-4e2e-a216-4275e3f5ee5b","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:00:40Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0d4162ab-aa3e-4e2e-a216-4275e3f5ee5b","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-09T00:00:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:01:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:01:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"096bfb0f-0a2e-4163-9c76-16fd6f164bb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:01:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:01:51Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:01:51Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b10628ad-e4fa-4fc6-a7f4-9338ee447f71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.createDefaultGroup"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.managePermissions"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f10e5b01-b765-4c1a-8b0a-a763b703e641"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f10e5b01-b765-4c1a-8b0a-a763b703e641"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f10e5b01-b765-4c1a-8b0a-a763b703e641","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f10e5b01-b765-4c1a-8b0a-a763b703e641","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:39Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"9fad891c-a494-4347-9904-3d224bbe12ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"6eec0e04-ef5d-402b-8de8-90e4a11b114f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"e6884401-1cf2-497f-b62f-e8a9e4ea20ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"f76ae453-1c80-40aa-b5fd-c63af8a7ef65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f10e5b01-b765-4c1a-8b0a-a763b703e641","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f10e5b01-b765-4c1a-8b0a-a763b703e641","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f10e5b01-b765-4c1a-8b0a-a763b703e641","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f10e5b01-b765-4c1a-8b0a-a763b703e641","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:02:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:02:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3a102bc6-4fc5-4536-930b-4e6f5970e118","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d8bd02fd-41ff-4c86-8950-53bba8e6817a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c88af23f-e1b3-46be-b0ff-7266f983029a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"a8ca492f-8063-4829-b1ef-e967a0beee15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf550f6f-c786-478f-bde8-8996c91adf28","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d5bc0d9f-4e16-41e0-a6c2-5e3fe51cc3cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"ebe74b1a-89c0-45c9-8704-55035e281f3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:02:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"d4fb9f4c-4ec1-4979-b50b-0053bd07006e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:41Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:42Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:42Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:42Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"3f0908a3-8533-40a2-9abc-1783387e0928","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:42Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:43Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:02:43Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:02:43Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:43Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"922e4a30-35ae-47aa-ba05-abb1ff8cb341","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:02:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-09T00:02:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:02:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c9a7a8c6-16a0-4d26-bd68-702ba248c4c0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:03:07Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328c74b8-a3ba-4077-8f85-e87d34639aa0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:03:22Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3e9eda0d-5220-4bc9-b214-c0c59d724e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:03:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:03:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a2501a64-bd32-4832-9072-5ef2b9e883ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:03:47Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:03:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:03:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"13f58caa-7bca-47cc-9630-b237d88e4d2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:04:07Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0b96ec0f-5579-4ddb-bbe2-f4913651d6cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:04:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:04:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"82f9e9fa-3438-496c-8467-c7049f699f09","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:04:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:04:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3929a0bc-81b0-4054-851d-c351463aaa12","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:04:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:05:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:06:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"c6994546-0ac6-4431-88db-39a2f12142fb"} {"level":"info","ts":"2026-07-09T00:06:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"b6f3b1f8-964a-4d7d-841f-e3526edb175e"} {"level":"info","ts":"2026-07-09T00:06:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"4dc02bc8-8c6b-4992-a530-0d4c1297020d"} {"level":"info","ts":"2026-07-09T00:06:37Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:06:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:06:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6a32bd96-31c4-4edf-a124-676b246785e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:06:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:06:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:06:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8c2ec9c5-ac4d-4a67-af0c-10e568f405a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:07:22Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c7bb485-fdd7-4536-a472-eb535abccb8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:07:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:07:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"589e1ce6-b3ae-41c5-afb8-cf396ae0b28c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:07:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:07:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2e1ad01e-699d-47b9-a2c4-ed5df3186683","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:08:00Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:08:22Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:08:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"edc0e5db-26cc-4708-a9ad-0cc204db93bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:08:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:08:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:08:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a90cb58b-4d3d-4a10-acf4-2af38959f62c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:09:23Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-09T00:09:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:09:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:09:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1b943c8b-4fe2-482d-9df7-de9ab4363a3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:10:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:10:07Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a27fb690-9cdd-4fae-8afd-25226c550019","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:10:22Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1dc6cc89-c1ea-4c68-9637-68010124a54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:10:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d79c194c-519d-4669-87e4-41565d033c1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:10:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ce46c9b-9f3c-47a7-95dc-b926de5c5b76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:10:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ae30aaa9-05a8-4849-8abc-9f8abee34988","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:10:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"033f2bca-fb2d-4d66-b3cc-68b6ae02c816","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd76b2fd-b214-439d-b1ec-db20e189b0e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"5bb13823-2cd2-434b-a1e7-3c0da73c62b3","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"5bb13823-2cd2-434b-a1e7-3c0da73c62b3","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"85031938-8928-40d0-95dd-a7616e5d475d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d036e959-a940-4747-bcdf-d4069fe8c412","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dab218d2-50d0-49b9-8d09-aaeabbb09810","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:10:39Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"138efa4f-ad77-43d2-90d2-6276f2a7fb1e","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:10:39Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"138efa4f-ad77-43d2-90d2-6276f2a7fb1e","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-09T00:10:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:10:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:10:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"22fb77de-a80a-4ffe-9904-348a47c792df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:11:22Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419"} {"level":"info","ts":"2026-07-09T00:11:22Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419"} {"level":"info","ts":"2026-07-09T00:11:22Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:11:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:11:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ce9b7cdd-ae01-4dcd-aa83-cac5d2e6f419","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:11:29Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:11:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:11:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c98bba23-f9c6-492c-8d88-1878b1ebb12b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:11:41Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-09T00:12:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:12:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-09T00:13:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:13:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:13:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4b3895c-90a6-45d6-aa8b-ec1aefd3b263","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:14:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:14:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"785b28f0-a4c6-4ffd-8827-7ddf9a54e02d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:14:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:14:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:14:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2dc7c28f-a372-4f32-a6fe-9b8de016c8aa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:15:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:15:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09664aa1-b3e2-4fc8-9808-d843ca6bd28d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:15:18Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:15:24Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:15:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:15:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4680f3c-b3c5-4d82-84a1-34eed761ac00","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:15:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:16:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:16:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bbda37-18c1-46a4-9535-6b309a919b6f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:16:28Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:16:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:16:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"053b9223-bcc2-442a-9dba-dabc1fd482d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:17:02Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:17:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:17:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e31acaae-96b8-4b5e-951f-af49c7c80e42","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:17:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:17:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:17:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"09ebc280-7d24-4404-a386-9b4661f1728e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:18:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3cdb8bd-ca7c-4dd4-b3e6-e96c66c5b336","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:18:09Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-09T00:18:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:18:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c2e99988-4854-45b6-833f-40fa226ba532","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:18:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e26df29b-3950-4ec3-a97e-77773f7530d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:18:47Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:18:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:18:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d827449-e9a1-4094-93c6-36a3d241b895","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:19:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:19:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:19:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18c09d7d-dccd-41ee-a603-36087cf69d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:20:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:20:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4a24631-6e1c-46ef-a1ad-37774b879d15","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:20:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6332d2ad-78c2-44ae-bb34-eac72202cd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d46a50de-3b0c-492a-818e-c980686a2fbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:20:39Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"41666d6d-d03b-4f5b-8add-7017c4770a0b","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:20:39Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"41666d6d-d03b-4f5b-8add-7017c4770a0b","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-09T00:20:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:20:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b"} {"level":"info","ts":"2026-07-09T00:20:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b"} {"level":"info","ts":"2026-07-09T00:20:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:20:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:20:54Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:20:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"76076262-ff1b-4c67-bec2-e8033cafc97b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:21:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:21:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b3befcef-9aab-41ed-86ee-3688cac67214","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:21:33Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:21:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:21:36Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e3d7eb7-9972-4f91-9443-420e24868d65","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:22:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:22:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e2ea161-cf24-4687-899a-8c81bd22b7f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:22:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51ac5af5-4249-4689-adf2-a98e0cfb549f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:22:54Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:22:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"42438436-03aa-48ff-903e-646f3cb49fc8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:23:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:23:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:23:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"febfcc5f-b330-4ebc-920e-724e3c874e7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:23:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:23:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2247ca6c-6f47-492a-bd32-2dce1dd21b17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:23:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:24:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b2bf39d3-6ae5-4dde-b244-381809e25f61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:24:20Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:24:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a805857b-352c-43a6-98e4-8281219f9ac7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:24:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:24:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a7876faa-e0e6-424a-b521-d3c1160182f8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:24:53Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-09T00:25:04Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:25:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7b0e6d62-c0e9-4d31-9186-d039ad5ada2c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:15Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:25:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d5f6b9f-d572-41fc-9da0-159c59c27f19","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:51Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"64c1c1fa-dd91-4ee4-a852-ccb9a7a7dfca","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:25:51Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"64c1c1fa-dd91-4ee4-a852-ccb9a7a7dfca","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:25:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.initializeModules"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.cleanupDisabledModules"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.provisionModules"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/helm.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.injectModuleEnv"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.updateModuleStatus"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01524e05-ebdf-47d2-8697-ed7df62e6317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2412df14-4fe5-413e-a135-d35c897cf99f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/dynamicownership.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:25:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0a921adb-394a-44c0-967c-c0f60a5e5760","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"de677ad7-fb22-44e6-bcf9-d731dc4b7b7a"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"de677ad7-fb22-44e6-bcf9-d731dc4b7b7a"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"de677ad7-fb22-44e6-bcf9-d731dc4b7b7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"de677ad7-fb22-44e6-bcf9-d731dc4b7b7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f99f4ad2-e1c2-465d-8a7a-931e97f3fa4d"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f99f4ad2-e1c2-465d-8a7a-931e97f3fa4d"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f99f4ad2-e1c2-465d-8a7a-931e97f3fa4d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f99f4ad2-e1c2-465d-8a7a-931e97f3fa4d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"edaf39e5-2943-4a5a-a1b2-7175cb430b74"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"edaf39e5-2943-4a5a-a1b2-7175cb430b74"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"3bc586db-2e8c-4c28-bf08-f8e4309fc755"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"3bc586db-2e8c-4c28-bf08-f8e4309fc755"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"4ff2abcc-26a1-46f9-aed5-c4fcd64d1612"} {"level":"info","ts":"2026-07-09T00:25:55Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"4ff2abcc-26a1-46f9-aed5-c4fcd64d1612"} {"level":"info","ts":"2026-07-09T00:25:56Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"4daef843-cb5d-49eb-ac98-91bad8b2ab58","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"4daef843-cb5d-49eb-ac98-91bad8b2ab58","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:25:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:25:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:25:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:25:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e5a4152-1cd5-4064-8f6e-c284f0400055","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9bc3f9cf-6964-4f11-aa2b-2f1152dd3b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm","msg":"delete","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47","gvk":"components.platform.opendatahub.io/v1alpha1, Kind=SparkOperator","ns":"","name":"default-sparkoperator"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e20c893-ea49-4c9e-ac10-67f83d762613"} {"level":"info","ts":"2026-07-09T00:25:58Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"727824ed-c568-405c-9df8-0657eb62c196","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"727824ed-c568-405c-9df8-0657eb62c196","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6b44db71-ee1a-4453-aa91-d4b784960f3a"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b4af7858-e73e-45e8-b0b3-c34f3372f924"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6178dd52-075c-4fb4-931f-747162551269"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6638ad7f-7286-44c1-8725-8cd4bf6016dc"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e4504a20-dfa9-48b3-b160-d902cf1e2ae2"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0f5ab96-da13-4de4-aaaf-fd72a61c92fa"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"fd13b33b-4f29-4142-ae43-7f23f1595d62"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a06d375e-598b-4c84-8bab-23ca59f86cca"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c52c358b-602d-4baa-80d3-5c36c433fa47","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84029943-7e94-4d99-a92b-8e4a8d8a37b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"de677ad7-fb22-44e6-bcf9-d731dc4b7b7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"de677ad7-fb22-44e6-bcf9-d731dc4b7b7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"de677ad7-fb22-44e6-bcf9-d731dc4b7b7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"de677ad7-fb22-44e6-bcf9-d731dc4b7b7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f1bc0168-74f5-4c80-b452-d7ebd78d6284"} {"level":"info","ts":"2026-07-09T00:25:58Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ba473a49-d0eb-4897-bc40-3f0357d2e77c"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:25:59Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"edaf39e5-2943-4a5a-a1b2-7175cb430b74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"dfe7f738-292b-444b-8034-711a2822bd63"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"edaf39e5-2943-4a5a-a1b2-7175cb430b74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"edaf39e5-2943-4a5a-a1b2-7175cb430b74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"924ebc4a-9d10-4414-a75b-c7159366e99b"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"3983e8cd-b165-48e8-bfb9-43ef6b5c920d"} {"level":"info","ts":"2026-07-09T00:25:59Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"947c50d9-1589-4f00-9f5f-ef88cb221b17"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f2430533-1e50-4ef1-9948-f5517d577368"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"4ff2abcc-26a1-46f9-aed5-c4fcd64d1612","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"9991f3b1-1c12-4e1e-8773-1e5883be015f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"30cc9b77-c9d6-409e-a0ce-d2857ba4cd47"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"3bc586db-2e8c-4c28-bf08-f8e4309fc755","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"3bc586db-2e8c-4c28-bf08-f8e4309fc755","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"3bc586db-2e8c-4c28-bf08-f8e4309fc755","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"52c3b857-3cd9-4768-9135-741cae587922","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"dbd5044f-a3e5-41fe-8de6-aa9c8f5a49d4","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"dbd5044f-a3e5-41fe-8de6-aa9c8f5a49d4","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe5121a-8ecc-47f2-8aea-ded07902f23a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"5976591e-5678-4608-95e8-72bd9fadc846","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"24194d51-5f93-49b8-8db8-0fc30b6cfe2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9aa93aaf-4537-4b11-8512-4fb9e3429fca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:00Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"e1070fb0-0d71-403a-9f28-1328b5bbeb3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:00Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"05c51436-d990-4d92-b43e-a2c36417c2bb","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"05c51436-d990-4d92-b43e-a2c36417c2bb","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0bbbe675-068d-4220-9c33-31c18bc99fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f99f4ad2-e1c2-465d-8a7a-931e97f3fa4d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f99f4ad2-e1c2-465d-8a7a-931e97f3fa4d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f99f4ad2-e1c2-465d-8a7a-931e97f3fa4d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"79e4d3b8-bbdb-4c88-b2fb-88376d2193a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bfbc5f49-e113-4373-8d5f-6746acabad5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-09T00:26:01Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"8c07789c-fd9b-4398-b939-70d5e692d966","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"8c07789c-fd9b-4398-b939-70d5e692d966","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"a3dfd7a9-eb70-49aa-9fae-29765799ecb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9a0b41f8-e8a8-4553-ba7e-5e156cb18960","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"4ff2abcc-26a1-46f9-aed5-c4fcd64d1612","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"4ff2abcc-26a1-46f9-aed5-c4fcd64d1612","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"4ff2abcc-26a1-46f9-aed5-c4fcd64d1612","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"4ff2abcc-26a1-46f9-aed5-c4fcd64d1612","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:26:01Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:01Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3d1dee78-495e-4e68-ac10-70612ca65e07","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d358a9c0-aa57-4cc6-b6ee-dc6eb0a80edb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-09T00:26:02Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"4ff2abcc-26a1-46f9-aed5-c4fcd64d1612","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"033cdbc1-0c88-4734-964f-ad105c3bc9e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:26:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:26:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-09T00:26:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-09T00:26:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"1ec24e46-ae25-4c9a-93d8-2d29957584c5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:04Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"382222c0-99f1-4aa3-8386-40b6ec3b5c01","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.initializeModules"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.cleanupDisabledModules"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.provisionModules"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/helm.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.injectModuleEnv"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.updateModuleStatus"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:05Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"39a743aa-c56d-4029-bed4-f1d59da69954","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"37e4a0f2-674c-42a6-ac8e-de68528df529","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/dynamicownership.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:05Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:26:06Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"fd09c41e-665c-4df0-825b-f7e47b46699d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"234afc75-758c-474e-8ecb-008ba09dbd57","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"7f4015c8-6cd7-4ff8-b7eb-6137a77c27ef"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"7f4015c8-6cd7-4ff8-b7eb-6137a77c27ef"} {"level":"info","ts":"2026-07-09T00:26:06Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"7f4015c8-6cd7-4ff8-b7eb-6137a77c27ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"2de9e0af-64a0-4829-ae5e-386618b3175b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"7f4015c8-6cd7-4ff8-b7eb-6137a77c27ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"d2d48034-8fd4-4331-8391-77c82532da69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ac84e855-2a47-4a95-a940-109f57e59dc3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"31108361-9ee4-4811-b19c-8b478382935b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"05c33108-bcb9-4fa6-9db5-a31cc4617c74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"7f4015c8-6cd7-4ff8-b7eb-6137a77c27ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"7f4015c8-6cd7-4ff8-b7eb-6137a77c27ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"7f4015c8-6cd7-4ff8-b7eb-6137a77c27ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"7f4015c8-6cd7-4ff8-b7eb-6137a77c27ef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"283bb642-814e-4126-9439-eafd6ca5c317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"9b88a807-c7cf-463d-addd-16284cfd4f7f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"d79ed389-5b36-40e7-bdd5-5149e8c3547d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"47057b66-255d-4f75-9705-f99d840c173a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:08Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"661b3fd5-700c-42e6-955c-ac74691dd0d4","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"661b3fd5-700c-42e6-955c-ac74691dd0d4","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b9b69fdc-ec5b-419c-8700-7678616d2301","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9a793d2e-d93b-45f0-b5c9-c6ad15fd5f80","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9a793d2e-d93b-45f0-b5c9-c6ad15fd5f80","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f28e76fc-c27d-497a-ac59-38de8db4aba0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ba905638-cf2d-491b-a42e-8a0b4378bc47","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:09Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:09Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:09Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:09Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:09Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a0b976f5-15f7-40cb-abf1-520c462c1608","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c41117a-cfc5-4725-8386-90c99b46b78f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-09T00:26:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"5cf895de-3419-4320-8328-76aec6b43206","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a9279b60-943e-4bb4-ab5c-d6487e4e0a76","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a9279b60-943e-4bb4-ab5c-d6487e4e0a76","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"cf27bd79-2e72-454d-814b-1bd8a614c375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:26:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9da3901e-dbfb-43d2-8815-dae3c0eccfb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:10Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"7be82a56-f67c-4e99-a459-f934e60a12b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"f5789586-69e5-4b00-8920-910a4eaee544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"51e36da3-1406-4f1b-b277-0d3ef9c3b234","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5019d908-ea30-4637-99ef-941c528bf1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f1d076-0712-4a18-ab05-4b9f2787aae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"531c115e-f226-487b-b85e-1a95ea0cb607","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"8807a7b5-d887-469f-88bc-bad5314f7729","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cb040f15-e1d6-4fab-9e75-bf33170bad4a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c3d9daf-4c68-4202-97e8-56e03ee1a489","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:22Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bd3168c3-39d1-44dd-92fd-b3a41ed8ae27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f851cccb-ab6c-4c7b-97b9-40d49b8a11df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3710fa82-64b9-4c93-9c4b-4b113d202d14","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:22Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"41997296-c887-4a7d-a811-77e6e309c327","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:22Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"41997296-c887-4a7d-a811-77e6e309c327","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5b0087b6-3847-4a1a-a07f-a1a35a006b45","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0d97c113-1e93-4448-9155-6a4bbeb7179c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cc0e8a2b-d026-4520-bd64-0686e6c15b48","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4b3ac11a-8b59-4602-9d75-14dddf81a253","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"84aa9581-c599-443b-810b-4dee8a5d84f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:26Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"6da77c5f-0109-4cb7-9659-10a945a2b374","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:26Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"6da77c5f-0109-4cb7-9659-10a945a2b374","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5fd7bb9b-e7e0-4529-9f46-b86dd4ea1b5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:29Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"4e860c11-4e8e-4887-aaef-fc29f4b76d6a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"abbd3b41-06fd-4a03-aa48-d2376757c7f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:29Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:29Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f37336f5-525f-4e9b-81f2-b77e682df9d8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:26:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"06f1df17-ba36-4974-9123-752f5f9c9af0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"546c4e7f-9ea1-4751-a8a2-6bba90f7135b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"30d042d3-2231-4239-bb13-983cf0e31152","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:38Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"018ef774-a3a8-4bc3-a3a7-ad6ca5b21d55","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:38Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"018ef774-a3a8-4bc3-a3a7-ad6ca5b21d55","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"54646f6d-a217-4778-8e1e-26560be2685a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6587971a-f4fb-4100-815e-ad2153de1401","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:45Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:45Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"de8b5520-2223-4cff-9aa8-41b6264678fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"690be100-919b-49c4-a2ee-45ab1c7c41ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:46Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"5ffb800f-ab53-404f-be8e-1610cb4a8d8d","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:46Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"5ffb800f-ab53-404f-be8e-1610cb4a8d8d","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c01c150b-df39-48ff-884f-855c6a14ff1a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:26:59Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a2a390c4-16d0-437e-bf9a-fba9c8ef4a76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fc35b93-37e2-4b9c-9bf4-6cc432771a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:26:59Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"1270e5d2-37a5-4ccb-9506-354c6f8f5232","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:26:59Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"1270e5d2-37a5-4ccb-9506-354c6f8f5232","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"20fbf24d-a784-45c1-b2b7-b87953b57f1c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"15e3b4f7-2f8e-41ba-83b3-9080a508bcbf","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"15e3b4f7-2f8e-41ba-83b3-9080a508bcbf","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:20Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8d990eec-eaea-4eb3-85bd-9bcdb895c3f0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fcf9b3ab-de98-4c17-8c95-90e72dcd569c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:27:30Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"54165e2e-e8de-433f-b0f1-3c468e2eef0f","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:27:30Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"54165e2e-e8de-433f-b0f1-3c468e2eef0f","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-09T00:27:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc4e4686-94b8-42e2-99d9-6f240843040a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:27:45Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b7a1bbb1-39cc-42b0-a97e-bd4d8cf9ac05","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:27:45Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b7a1bbb1-39cc-42b0-a97e-bd4d8cf9ac05","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d10ea1d3-602f-45e4-bbb3-bdbd32fe91e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"af3a8b48-44e9-4f32-aa34-d61d98625905","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"af3a8b48-44e9-4f32-aa34-d61d98625905","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:27:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e75f5c34-b5cc-49f8-acc7-2b0aac568471","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:28:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"670b9dbe-127a-4014-a6dd-426acee63876","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b900275b-4897-4120-b381-3c33004f8739","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b900275b-4897-4120-b381-3c33004f8739","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8619a111-c214-48bd-b06e-3e2ad9e93f5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:28:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6dd4e752-cb63-4870-8c40-9e66a9e2a08d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:28:49Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"8ff5adcf-1400-49b9-8441-a3807acae585","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:28:49Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"8ff5adcf-1400-49b9-8441-a3807acae585","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee3676f0-4976-4f15-83ea-3944c2438d88","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:04Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"455863c1-9efd-4082-ad0e-a901ec2739e6","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:29:04Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"455863c1-9efd-4082-ad0e-a901ec2739e6","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5ed1427e-7494-48ba-b098-bf9d7590d360","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:17Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a33fa914-b82c-41a0-9c6e-60e44d47e636","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a33fa914-b82c-41a0-9c6e-60e44d47e636","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"10e003ae-c90e-49f7-b248-15da04c8df0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7e67a973-5222-4fe0-93f7-8865ffcd5f29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"16710205-713c-442b-b520-4db073a46935","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4636b0d4-c691-418d-8282-804dd7252eac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"367c5cb3-b393-43bf-9c2f-7192774668a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5c107ab9-4cfa-4bbe-a83f-c5183944ea99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7491c9c6-f5fd-45ee-b3e3-e55098a284d5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6311ca10-54aa-4bc7-8c3e-8f84ae797b22","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:29:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"d524f442-f8a3-492b-86ca-ff6883a19e8e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"59c1e906-4903-42ab-9f7c-5c6fb4be998e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2e0d2ac6-6c23-4a95-80df-09f0971ca3c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2e86b099-62d3-4650-a700-ec833282ddc1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:27Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0f87ebb2-43bf-44a4-aa86-d9f38b10f322","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:29:27Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0f87ebb2-43bf-44a4-aa86-d9f38b10f322","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-09T00:29:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e17905d9-389b-456b-bc2e-378c1f9226e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:29:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6abd4c21-440e-4999-9d23-fae374b6d863","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd49bb86-76a2-408b-bfc8-602ffa7cf6fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:40Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a3d611d9-727f-4a09-b81e-fd2f6d194fe2","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:29:40Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a3d611d9-727f-4a09-b81e-fd2f6d194fe2","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c5844059-10f4-4b0d-bd92-4eeee2d286e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:29:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e97c1ee-04ba-4fc8-8bb4-88cb186b0cd0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"05a427d1-fafc-4250-892a-bb857832efdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aaa9aece-2a4d-4439-90c0-5c9025aa49e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ebc40c4b-9c2f-463b-b7e8-c7fab013ae1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:43Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2f04697c-8048-4567-ba10-5acd7d6f7867","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:29:43Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2f04697c-8048-4567-ba10-5acd7d6f7867","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"24f9b64e-d1c1-455f-8f8d-0a019f73606a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:29:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"792990bb-2f4d-4470-9d40-cd45ae3d2334","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"41c6cb3f-22fe-4d7c-8f75-65c4ed1db421","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:55Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"fdc7a9e9-4dfd-4a6a-b547-332cfffb0344","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:29:55Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"fdc7a9e9-4dfd-4a6a-b547-332cfffb0344","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"reconcile","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"apply","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.initializeModules"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.cleanupDisabledModules"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.provisionModules"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/helm.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.injectModuleEnv"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.updateModuleStatus"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"020ec110-3d08-47bb-9f53-fb031ba2e0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e54959d8-e2fb-4a54-a21f-37503495d439","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/dynamicownership.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"db1743d6-00ba-42fe-82ce-0cba13cb7886","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"9d4dbdaf-1a06-4cfb-bfb2-668b185d3753"} {"level":"info","ts":"2026-07-09T00:29:58Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"9d4dbdaf-1a06-4cfb-bfb2-668b185d3753"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:29:59Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"72266305-fc6c-46f6-90a6-6bc8ebefff24","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b40dc9fe-6112-437e-9a14-347321394314"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b40dc9fe-6112-437e-9a14-347321394314"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b40dc9fe-6112-437e-9a14-347321394314","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"11944924-de0b-4c75-a0b0-817c4643f2e0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"136a375e-2ebc-459b-b13b-629847bef885","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"801b6049-7bb0-406f-92a8-df4ba7a13042","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"801b6049-7bb0-406f-92a8-df4ba7a13042","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0879b6bf-6fcb-4927-aaf8-9304d870275d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"94db2262-170c-49c0-8d27-15dad55d510f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"70bf2bb5-e577-46da-be1f-54183bab5091"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"70bf2bb5-e577-46da-be1f-54183bab5091"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"70bf2bb5-e577-46da-be1f-54183bab5091","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"70bf2bb5-e577-46da-be1f-54183bab5091","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fab8a2c0-73c0-44b1-81fb-8e6cac00535c"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fab8a2c0-73c0-44b1-81fb-8e6cac00535c"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"45cc8984-9a98-41ad-bdcb-2163719988ce"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"45cc8984-9a98-41ad-bdcb-2163719988ce"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"12090940-9804-4819-a21c-e9c7bbf02513"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"12090940-9804-4819-a21c-e9c7bbf02513"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"67833725-10cf-4df3-8c0a-380aae59470b"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"67833725-10cf-4df3-8c0a-380aae59470b"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5e0b32f2-ada5-4467-b898-3bde0d7c382b"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5e0b32f2-ada5-4467-b898-3bde0d7c382b"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5e0b32f2-ada5-4467-b898-3bde0d7c382b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5e0b32f2-ada5-4467-b898-3bde0d7c382b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5e0b32f2-ada5-4467-b898-3bde0d7c382b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:29:59Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b40dc9fe-6112-437e-9a14-347321394314","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b40dc9fe-6112-437e-9a14-347321394314","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b40dc9fe-6112-437e-9a14-347321394314","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"12090940-9804-4819-a21c-e9c7bbf02513","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"12090940-9804-4819-a21c-e9c7bbf02513","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"12090940-9804-4819-a21c-e9c7bbf02513","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"9d4dbdaf-1a06-4cfb-bfb2-668b185d3753","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"67833725-10cf-4df3-8c0a-380aae59470b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"9d4dbdaf-1a06-4cfb-bfb2-668b185d3753","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"9d4dbdaf-1a06-4cfb-bfb2-668b185d3753","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"9d4dbdaf-1a06-4cfb-bfb2-668b185d3753","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"12090940-9804-4819-a21c-e9c7bbf02513","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"12090940-9804-4819-a21c-e9c7bbf02513","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:00Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"12090940-9804-4819-a21c-e9c7bbf02513","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"67833725-10cf-4df3-8c0a-380aae59470b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"67833725-10cf-4df3-8c0a-380aae59470b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"45cc8984-9a98-41ad-bdcb-2163719988ce","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"45cc8984-9a98-41ad-bdcb-2163719988ce","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"45cc8984-9a98-41ad-bdcb-2163719988ce","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"45cc8984-9a98-41ad-bdcb-2163719988ce","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm","msg":"delete","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91","gvk":"components.platform.opendatahub.io/v1alpha1, Kind=SparkOperator","ns":"","name":"default-sparkoperator"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cf9f41f1-2f46-4b1b-8f91-01c22bdbd1f8"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bf9bfabd-63a1-4ff1-b467-ada9552a4b8b"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6e55fe7c-039b-4cc9-bcfb-e8e0d67f353b"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a18d9013-9d27-4d7c-89c0-646ec5f182cb"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ae0ee840-1674-4c85-ae4a-f7059177789e"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a781123e-654c-4d4c-82af-7a7cc8f0fefc"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"29fd5f3d-7a41-4b66-84f9-4987093ed4c3"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"dac8042c-931f-4118-8209-ebb975a64b2a"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"61b9eced-dcb2-45e6-b875-c09e7e270501"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a286bc80-e925-43d6-8aed-a3b153f4932d"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"525c2fcc-292d-449a-8f6a-4a63ce643339"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6c2f436d-365d-4853-92f2-1a61228f46b4"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b28849aa-aa78-455f-8510-e1de215d88f0"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c3e828a6-5160-4e9f-b441-7f3a51328092"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"40fe984c-eba2-45c7-a1d5-d842a741bb6c"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0a1c6f5-eab2-4a28-b3d0-ae18a02ae8de"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f6026503-bd12-45ed-bb62-5a98dadae227"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ce96f21e-06fa-4e4b-b2ec-176e95ce13ad"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"37de9f03-858b-41ec-b48f-fc889f885fee"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6f1803d2-929a-4761-9496-689351739a95"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"32d1bd6a-e9fa-42d9-98cc-d38ea4135e97"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"297ec01d-560f-4c6f-b3f9-4beb90cd8983"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:01Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"260dd52c-4f46-4dae-981b-e47903c88297","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"260dd52c-4f46-4dae-981b-e47903c88297","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2fc0aa19-0557-4b88-8a6a-0b6a7b88a911"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b17795c2-f00c-4a73-b6df-a813ce9a991c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f34d0569-1f8f-49a4-b3f4-0a57739fdc19"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81b040a7-cd98-4d06-a49a-66a245b7b5e7"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"cf2bc529-6852-485a-8555-8ea2f2f75b91","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f3f9e516-216b-4162-bac2-de955a98dda6"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:01Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ca95cc27-9e21-4989-bb30-f7c956b3cafb"} {"level":"info","ts":"2026-07-09T00:30:02Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"587bd300-b3c6-4ab9-86a5-6e4ead1ef8ca","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"587bd300-b3c6-4ab9-86a5-6e4ead1ef8ca","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6ab5b610-3486-4fb2-b590-d4d747783125","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"64fc9067-ce39-4a6f-b5e7-db49083d08fc"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"195ebe54-4e33-436a-b659-580c9bdc1129"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a20b2ff5-8038-416a-80bc-8c0452219352","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a20b2ff5-8038-416a-80bc-8c0452219352","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c21fb74a-0ad9-41df-8b5b-35ab9e699c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:02Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc9a9aca-15cc-4bbd-8e4a-fa651ecb716d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a6da54ab-5bae-410c-a789-c7e4f65b8df2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4096ff99-1d14-4600-95a0-29e397d546f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-09T00:30:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b55e1625-bb3c-4e80-9fca-4df34679b7cb","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b55e1625-bb3c-4e80-9fca-4df34679b7cb","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"6226348e-6009-4798-96e2-cf505480e242","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:30:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:03Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"88b28d05-0785-4e94-a117-1f857fef96e2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"93b5f884-3a3a-4ad8-a6a8-153da5385f9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d7f55e6e-5108-4590-b526-829a3c69fa6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-09T00:30:04Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"286d9866-0165-48b0-b410-27fc1b926217","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-09T00:30:04Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-09T00:30:04Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"044cb60b-2572-43b0-935c-46d7a2e60e64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:05Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b47e8a2a-a161-4039-bc37-455867d8ba0e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:30:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:30:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:30:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"708f8694-6ded-4841-8beb-7d8ac495bf0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:30:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd63c0aa-f73f-4e9f-a477-5a92e6ea422c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:30:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2f239732-466c-4bb9-a333-c501cd70f336","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"2bef0dd6-3e38-4224-aa8d-2f9ad9f42120","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:40Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"3ab0f767-0723-4664-931a-3bbd642df6d4","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:40Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"3ab0f767-0723-4664-931a-3bbd642df6d4","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"reconcile","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"apply","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.initializeModules"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.cleanupDisabledModules"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.provisionModules"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/helm.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.injectModuleEnv"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.updateModuleStatus"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d"} {"level":"info","ts":"2026-07-09T00:30:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:43Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"81282679-0100-4a69-8ffb-756d03e5284d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"131ed71b-a13a-4af5-80d0-3f5553bbfb5b"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"131ed71b-a13a-4af5-80d0-3f5553bbfb5b"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"131ed71b-a13a-4af5-80d0-3f5553bbfb5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c9f08087-2405-4a38-8294-e7376850b844","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/dynamicownership.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"131ed71b-a13a-4af5-80d0-3f5553bbfb5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"5afe8e34-c754-4661-b73b-8fe4dac0d29f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c"} {"level":"info","ts":"2026-07-09T00:30:44Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"442889a5-7976-49bd-bc9b-33ef64e6d7c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"244033af-2e22-4f15-bf60-76c6f0090e1e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"dd2c9609-b6dd-4cbe-9dec-c67a507e3949","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"131ed71b-a13a-4af5-80d0-3f5553bbfb5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"131ed71b-a13a-4af5-80d0-3f5553bbfb5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"131ed71b-a13a-4af5-80d0-3f5553bbfb5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"131ed71b-a13a-4af5-80d0-3f5553bbfb5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:30:45Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"82823666-70c7-4928-ae0f-ac23501a16c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3aab91ee-2b75-4e69-a00a-0adc1c564048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:45Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e5e35b02-6875-45a5-8114-d097667a839f","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e5e35b02-6875-45a5-8114-d097667a839f","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"58ef66dc-9e15-4cad-b8d3-7bff4c73e16e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"a2b9257a-c36e-43db-bd80-adb8c8cb73cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0927152e-560d-4f95-bb80-4a5b07d7127d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"67fdab54-2d74-4c18-8837-2cf483f53be0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"0d7a1e73-5085-4bf1-a24f-e3f7e7fde254","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:47Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"80e61729-149a-44af-bd25-384e1bbe98c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"647be50e-57c8-4889-b689-3fb572ba401c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"6d8406e3-7e77-4a81-bbc6-d2013149e964","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:48Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e1a3ff86-f4bc-4983-962c-d27dbc283266","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e1a3ff86-f4bc-4983-962c-d27dbc283266","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"840e3bf7-ab8b-4549-ab06-475fc779ca35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e6b748ea-219b-41c9-99ae-cd57d7e2a6bf","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e6b748ea-219b-41c9-99ae-cd57d7e2a6bf","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"92dd1236-f586-4d81-af78-cf747731f34c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4928a6ce-42f9-4aac-be62-d172936972dd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"71f3d061-fd3d-4624-9100-d2e94dd9069f","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"71f3d061-fd3d-4624-9100-d2e94dd9069f","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-09T00:30:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9bb2c4e2-4352-4834-aed9-d7201c95f56b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-09T00:30:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1d1dd120-e0b4-4ee3-b5f0-6953c3ec2df5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:49Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"2f368176-d5a8-484d-a71c-afd0edc6afaf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b1c13e78-1e1f-409b-aa02-11dcb3b9ed95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"50ff8d4f-1e97-4f8c-a160-dc9bb03bc5cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b004bc82-8e74-4e29-a52b-9c260c7d1d71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"85dbb382-2455-4e6e-922c-c73375447d19","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"b9fc12a5-d26a-4b84-928b-1cf691a2736e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:30:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:30:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"aeec03c0-5d1c-479f-9498-3b245eb3ce2b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:30:59Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8aa0c2fe-b22d-4646-adf4-b45ad0428976","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:31:01Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:01Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2423dd05-e8c8-4b22-aaac-2a07b5886375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bb094de3-473a-41b1-b414-e7aa25c6ade8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:31:02Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ae5f5fdf-03dd-41c5-9d6d-037e5b058358","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"45a1fbfe-fd25-4a64-ad25-70080cb7b1b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:31:02Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"99f495a5-c768-4e4a-bedb-e50a58a0d975","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-09T00:31:02Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"99f495a5-c768-4e4a-bedb-e50a58a0d975","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"093a28e1-4be7-47fb-a229-989cfeed38d9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:31:07Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2b01f185-656a-40e5-8091-dcc75cac45b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:31:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bb69eb71-e4a6-4f5d-ab69-b4a3ad35240c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:31:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:31:49Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-09T00:32:35Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:32:57Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-cz8y8","reconcileID":"d4f03c9e-3b24-4671-8d0a-5fe8002c297f"} {"level":"info","ts":"2026-07-09T00:32:58Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-cz8y8","reconcileID":"bacee67a-4915-4c30-8402-a7815be48be4"} {"level":"info","ts":"2026-07-09T00:32:58Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-cz8y8","reconcileID":"a533938d-aa2d-40c9-9a0b-2d03e6dc440e"} {"level":"info","ts":"2026-07-09T00:32:58Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-orh1u","reconcileID":"54daf8d0-e6e2-4cc5-855b-2f1da1134cc2"} {"level":"info","ts":"2026-07-09T00:32:58Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-orh1u","reconcileID":"c711b92a-f96a-4411-920d-1654b63eeebe"} {"level":"info","ts":"2026-07-09T00:32:58Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-orh1u","reconcileID":"e92b7794-0e97-4ae7-a852-53a1a3768aab"} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2"} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2"} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-09T00:33:01Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-09T00:33:01Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e5ad4a58-b483-44e5-a047-9f1b6fe4c1b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:33:02Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74ac7551-7d1f-4741-a15f-69eba4364246","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-09T00:33:02Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-09T00:33:02Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"fe86ac2c-98eb-4c59-9729-3dd1c423aea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-09T00:33:03Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:03Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e02c72e5-975c-4e19-8d92-2d016a0abeac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f9c94ff4-ca44-4c29-8cd2-2b4794862b8f"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f9c94ff4-ca44-4c29-8cd2-2b4794862b8f"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f9c94ff4-ca44-4c29-8cd2-2b4794862b8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f9c94ff4-ca44-4c29-8cd2-2b4794862b8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:33:22Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282"} {"level":"info","ts":"2026-07-09T00:33:23Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"reconcile","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"apply","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.initialize"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.createDefaultGroup"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.managePermissions"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:24Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"3775ebf5-6a44-4638-bad6-955dbb184631","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"87e2a75f-9a0b-4749-8c69-097dfb916fc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bc9a5276-7164-4a87-ba15-1d7a2801ed93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"b22c1f0d-2a3d-42ee-aedd-770971b237b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"8325f9a1-cb13-4dc1-9fa7-75d004223aa3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ad5d3ef6-b46b-4fd1-9240-f57fe6fdd3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"18372fb1-fa18-4866-8e36-e978c8c0bf61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"717f2b91-42de-4775-a581-e48dee6d3fb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"7ce4c783-6f7a-4bac-ba5e-6586797600ea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"fb710a7b-26d1-46d6-a186-ca3313a4346b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"62ea14c9-e28d-4a11-a885-7fab7424b992","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"b9e8f49f-3aba-4ee0-8e60-ef8653356bd6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"521ee979-ce0d-4b38-8383-669a24947282","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"c41b5843-7141-4318-9fb6-d81720ceb1c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f9c94ff4-ca44-4c29-8cd2-2b4794862b8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f9c94ff4-ca44-4c29-8cd2-2b4794862b8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f9c94ff4-ca44-4c29-8cd2-2b4794862b8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f9c94ff4-ca44-4c29-8cd2-2b4794862b8f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:25Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b625b76c-11f9-4bb3-abc5-c4e63de6ee35","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"a2645366-5888-4d91-b8a4-c9322d82345c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"5e85db27-0c7c-4c6d-b391-3a89bf9916a5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"39f62f6a-76a4-4fe5-aa8f-5ec0f3e6fcd4"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"39f62f6a-76a4-4fe5-aa8f-5ec0f3e6fcd4"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"39f62f6a-76a4-4fe5-aa8f-5ec0f3e6fcd4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ea6e19c9-8781-4872-98b3-f2ef62b86c5e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"39f62f6a-76a4-4fe5-aa8f-5ec0f3e6fcd4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"ce40be84-dd6a-4d35-943b-76c9b7452dea","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"36c6cbca-eb47-4588-836b-f0cec0b6ff62","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"559791fe-a84c-4369-9306-445a4a7105ff","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"83cdf5e4-24f6-463a-9741-8efd52428db8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"39f62f6a-76a4-4fe5-aa8f-5ec0f3e6fcd4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"39f62f6a-76a4-4fe5-aa8f-5ec0f3e6fcd4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"39f62f6a-76a4-4fe5-aa8f-5ec0f3e6fcd4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"39f62f6a-76a4-4fe5-aa8f-5ec0f3e6fcd4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d6117a47-e1d9-4c73-9d7d-6f44237364d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"852d9740-40b6-48d1-86cc-1692439447a4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"reconcile","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"apply","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.initialize"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.createDefaultGroup"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.managePermissions"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:26Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"e3f6f587-d3c9-48e7-acf5-e8977bd7a3df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:27Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"96459a06-5b56-49fb-8c10-6e63b3f449ba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-09T00:33:27Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"9c32e1c2-af07-4141-85d5-d7c8792f7ad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"a0f4fc5c-5e5f-428b-a5d6-7b841304eac2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:28Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-09T00:33:29Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"506dbf6c-e9bd-4b16-a0da-7415522f3e3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-09T00:33:30Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"ed322e44-5d46-4dee-86ae-07a1a0da753d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} ==================== OPERATOR POD LOG: rhods-operator-6f6bb89fdf-f4zpw.log ==================== ==================== OPERATOR POD LOG: rhods-operator-6f6bb89fdf-kr2pd.log ==================== ==================== WORKLOAD POD LOG: redhat-ods-applications/dashboard-redirect-6bd46884d4-d2rg2.log ==================== 2026/07/08 22:57:24 [notice] 1#1: using the "epoll" event method 2026/07/08 22:57:24 [notice] 1#1: nginx/1.26.3 2026/07/08 22:57:24 [notice] 1#1: built by gcc 11.5.0 20240719 (Red Hat 11.5.0-14) (GCC) 2026/07/08 22:57:24 [notice] 1#1: OS: Linux 5.14.0-570.60.1.el9_6.x86_64 2026/07/08 22:57:24 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576 2026/07/08 22:57:24 [notice] 1#1: start worker processes 2026/07/08 22:57:24 [notice] 1#1: start worker process 12 2026/07/08 22:57:24 [notice] 1#1: start worker process 13 ==================== WORKLOAD POD LOG: redhat-ods-applications/data-science-pipelines-operator-controller-manager-7fc7f76glsfg.log ==================== 2026-07-08T22:57:26Z INFO setup starting manager 2026-07-08T22:57:26Z INFO controller-runtime.metrics Starting metrics server 2026-07-08T22:57:26Z INFO starting server {"name": "health probe", "addr": "[::]:8081"} 2026-07-08T22:57:26Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I0708 22:57:26.373141 1 leaderelection.go:258] "Attempting to acquire leader lease..." lock="redhat-ods-applications/f9eb95d5.opendatahub.io" I0708 22:58:28.966920 1 leaderelection.go:272] "Successfully acquired lease" lock="redhat-ods-applications/f9eb95d5.opendatahub.io" 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Secret"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Deployment"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Secret"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.DataSciencePipelinesApplication"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.ConfigMap"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Service"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Route"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.ConfigMap"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Pod"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.ServiceAccount"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.PersistentVolumeClaim"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.NetworkPolicy"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Role"} 2026-07-08T22:58:28Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.RoleBinding"} 2026-07-08T22:58:29Z INFO Starting Controller {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication"} 2026-07-08T22:58:29Z INFO Starting workers {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "worker count": 10} 2026-07-08T23:12:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:04Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:04Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:04Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:04Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:04Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:04Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:04Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:04Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:05Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:05Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:05Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:05Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:05Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:05Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:14Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:14Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:14Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:14Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:14Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:14Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:14Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:14Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:15Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:15Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:15Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:15Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:15Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Unable to connect to Database: dial tcp 172.30.26.71:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:15Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:15Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:15Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:12:15Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:26Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:26Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:27Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:27Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:27Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:27Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:27Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:27Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:27Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:27Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:28Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:28Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:28Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:28Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:28Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:28Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:28Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:28Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:28Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:29Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:29Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:29Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:29Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:29Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:29Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:29Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:30Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:30Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:30Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:30Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:30Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:30Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:30Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:30Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:31Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:31Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:31Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:31Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:31Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:31Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:31Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:31Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:33Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:33Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:33Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:33Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:33Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:33Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:33Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:33Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:34Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:34Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:34Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:34Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:34Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:34Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:34Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:34Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:35Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:37Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:37Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:38Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:38Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:38Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:38Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:12:38Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:38Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:42Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:12:43Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:12:43Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:13:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:20Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:13:20Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:20Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:13:20Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:13:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:20Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:13:20Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:20Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:13:20Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:13:20Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:20Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:13:20Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:20Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:13:20Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:13:20Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:20Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:13:20Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:20Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:13:20Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:13:20Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:20Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:13:21Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:13:21Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:13:21Z INFO DSPA resources cleaned up. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:21Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:13:21Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:13:21Z INFO DataSciencePipelinesApplication.datasciencepipelinesapplications.opendatahub.io "dspa" not found {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:13:21Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "8de51990-92db-496c-9b0d-a723952e07c1", "error": "Operation cannot be fulfilled on datasciencepipelinesapplications.datasciencepipelinesapplications.opendatahub.io \"dspa\": StorageError: invalid object, Code: 4, Key: /kubernetes.io/datasciencepipelinesapplications.opendatahub.io/datasciencepipelinesapplications/cy-e2e-automl-binary-522120/dspa, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 11cb5b0e-cfea-4db0-9493-bd339ac199de, UID in object meta: "} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:14:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:00Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:00Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:00Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:00Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:00Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:00Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:00Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:00Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:00Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:00Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:00Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:00Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:00Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:00Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:01Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:01Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:01Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:01Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:01Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:01Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:01Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:01Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:07Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:07Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:07Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Unable to connect to Database: dial tcp 172.30.157.237:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:07Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:07Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:07Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:07Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:07Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:14:18Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:18Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:18Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:18Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:19Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:20Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:20Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:20Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:20Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:20Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:20Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:20Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:20Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:21Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:22Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:24Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:25Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:25Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:25Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:25Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:25Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:25Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:25Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:25Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:25Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:26Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:26Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:26Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:26Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:26Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:26Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:26Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:26Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:26Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:27Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:27Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:27Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:27Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:27Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:27Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:27Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:27Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:27Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:27Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:27Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:27Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:27Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:27Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:27Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:27Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:27Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:28Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:28Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:28Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:28Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:28Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:28Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:28Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:28Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:31Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:31Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:31Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:14:31Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:31Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:34Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:35Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:14:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:14:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-08T23:15:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:15:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:15:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:15:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:15:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:15:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:15:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:15:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:15:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:15:23Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:15:23Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:15:23Z INFO DSPA resources cleaned up. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-08T23:23:52Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:52Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:52Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:23:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:23:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:23:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:23:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:23:54Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:23:55Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:23:55Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:23:59Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:23:59Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:59Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:59Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:59Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:59Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:59Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:23:59Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:23:59Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:24:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:24:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:24:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:24:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:00Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:00Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:24:11Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:11Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:11Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:24:11Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:11Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:11Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:11Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:11Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:11Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:24:31Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:31Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:31Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:31Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:31Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:31Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:31Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:31Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:31Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:51Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:24:51Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:24:51Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:51Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:51Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:51Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:51Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:24:51Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:51Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:24:51Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:25:11Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:11Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:11Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:11Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:11Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:11Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:11Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:11Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:11Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:25:31Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:31Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:31Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:31Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:31Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:31Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:31Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:31Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:31Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:51Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:25:51Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:25:51Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:51Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:51Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:51Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:51Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:25:51Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:51Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:25:51Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:11Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:26:12Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:26:12Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:26:12Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:12Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:12Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:12Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:26:32Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:32Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:32Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:32Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:32Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:32Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:32Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:32Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:32Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:52Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:26:52Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:26:52Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:52Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:52Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:52Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:52Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:52Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:26:52Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:26:52Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:27:12Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:27:12Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:27:12Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:12Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:12Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:12Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:27:32Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:27:32Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:27:32Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:32Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:32Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:32Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:52Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:52Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:52Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:52Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:27:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:27:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:27:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:27:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:28:13Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:13Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:13Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:13Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:13Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:13Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:13Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:13Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:13Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:28:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:28:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:28:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:28:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:28:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:29:13Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:13Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:13Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:13Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:13Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:13Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:13Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:13Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:13Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:29:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:29:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:29:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:29:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:29:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:30:13Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:13Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:13Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:13Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:13Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:13Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:13Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:13Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:13Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:30:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:30:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:30:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:30:54Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:30:54Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:30:54Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:30:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:30:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:30:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:14Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:14Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:14Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:14Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:14Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:14Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:14Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:14Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:14Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:14Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:14Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:14Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:34Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:34Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:34Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:34Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:34Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:34Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:34Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:34Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:34Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:54Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:54Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:55Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:55Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:55Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:55Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:57Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:31:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:31:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:31:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:31:57Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:31:57Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:31:57Z INFO DSPA resources cleaned up. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:09Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"test-dsp-custom-pip-prj-150747"}, "namespace": "test-dsp-custom-pip-prj-150747", "name": "dspa", "reconcileID": "4433c805-7dc4-45ac-a48e-2b8e3e85b8a3", "error": "Operation cannot be fulfilled on persistentvolumeclaims \"mariadb-dspa\": the object has been modified; please apply your changes to the latest version and try again"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:09Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:09Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:09Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:09Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:09Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:09Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:09Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:10Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:10Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:10Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:10Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:10Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:10Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:10Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:10Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:10Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:11Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:11Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:11Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:11Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:11Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:11Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:11Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:11Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:21Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:21Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:21Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:21Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:22Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:22Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:22Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:22Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:22Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:22Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:22Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:22Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:22Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:32:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:32:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:32:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:32:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:33:13Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:13Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:13Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:13Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:13Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:13Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:13Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:13Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:13Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:33:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:33:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:33:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:33:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:33:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:34:13Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:13Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:13Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:13Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:13Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:13Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:13Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:13Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:13Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:34:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:34:53Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:34:53Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:53Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:53Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:53Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:53Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:53Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:34:53Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:34:53Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:35:14Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:14Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:14Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:14Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:14Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:14Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:14Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:14Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:14Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:35:34Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:35:34Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:35:34Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:34Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:34Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:34Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:35:54Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:35:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:35:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:35:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:36:14Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:14Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:14Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:14Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:14Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:14Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:14Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:14Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:14Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:36:34Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:34Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:34Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:34Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:34Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:34Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:34Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:34Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:34Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:36:54Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:36:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:36:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:36:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:37:14Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:14Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:14Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:14Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:14Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:14Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:14Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:14Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:14Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:37:34Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:34Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:34Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:34Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:34Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:34Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:34Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:34Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:34Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:37:54Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:37:54Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:54Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:54Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:54Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:54Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:37:54Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:54Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:37:54Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:38:15Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:15Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:15Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:15Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:15Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:15Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:15Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:15Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:15Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:38:35Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:38:35Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:38:35Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:35Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:35Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:35Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:38:55Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:38:55Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:38:55Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:38:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:38:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:38:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:15Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:15Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:15Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:15Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:15Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:15Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:15Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:15Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:15Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:35Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:35Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:35Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:35Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:35Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:35Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:35Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:35Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:35Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:55Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:55Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:55Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:55Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:55Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:55Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:55Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:56Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:56Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:56Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:56Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:56Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:56Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:56Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:57Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:57Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:57Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:57Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:58Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:58Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:58Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:58Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:58Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:58Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:39:58Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:39:58Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:58Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:39:58Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:39:58Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:39:58Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:39:58Z INFO DSPA resources cleaned up. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:57Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:57Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:57Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:57Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:57Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:57Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"test-pipelines-prj-643148"}, "namespace": "test-pipelines-prj-643148", "name": "dspa", "reconcileID": "ee91da44-f07d-45d7-85fa-c09bec3d35d2", "error": "Operation cannot be fulfilled on services \"mariadb-dspa\": the object has been modified; please apply your changes to the latest version and try again"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:57Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:57Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:57Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:57Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:57Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:57Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:58Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:58Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:58Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:58Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:58Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:58Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:40:58Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:40:58Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:40:58Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:40:58Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:00Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:00Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:00Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:00Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:00Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:00Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:00Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:00Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:00Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:00Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:10Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:10Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:10Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:10Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:10Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:10Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:10Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:10Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:10Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:11Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:11Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:11Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:11Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:11Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:11Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:11Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:11Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:11Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:11Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:11Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:22Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:22Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:22Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:22Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:22Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:22Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:22Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:22Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:22Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:22Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:42Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:42Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:42Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:42Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:42Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:42Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:41:42Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:41:42Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:42Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:41:42Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:41:42Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:42:02Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:42:22Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:22Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:22Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:22Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:22Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:22Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:22Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:22Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:22Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:42:42Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:42:42Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:42Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:42:42Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:42Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:42Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:42Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:42Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:42:42Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:43:02Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:43:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:43:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:43:23Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:23Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:23Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:23Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:23Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:23Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:23Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:23Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:23Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:43:43Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:43:43Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:43Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:43:43Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:43Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:43Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:43Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:43Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:43:43Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:44:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:44:23Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:23Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:23Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:23Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:23Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:23Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:23Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:23Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:23Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:44:43Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:44:43Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:43Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:43Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:43Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:43Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:44:43Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:43Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:44:43Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:45:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:45:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:45:04Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:04Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:04Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:04Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:04Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:04Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:04Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:04Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:04Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:45:24Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:24Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:24Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:24Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:24Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:24Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:24Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:24Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:24Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:45:44Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:45:44Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:44Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:44Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:44Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:44Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:45:44Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:44Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:45:44Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:46:04Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:04Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:46:04Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:04Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:04Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:04Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:04Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:46:04Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:46:04Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:46:24Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:24Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:46:24Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:24Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:24Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:24Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:24Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:46:24Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:24Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:46:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:46:44Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:46:44Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:46:44Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:46:44Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:44Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:44Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:44Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:44Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:46:44Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:47:04Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:04Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:04Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:47:04Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:04Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:04Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:04Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:04Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:47:05Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:05Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:05Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:05Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:05Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:05Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:05Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:05Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:05Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:25Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:47:25Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:25Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:25Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:25Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:25Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:25Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:25Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:25Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:25Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:47:45Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:47:45Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:45Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:45Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:45Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:45Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:47:45Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:45Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:47:45Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:48:05Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:05Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:48:05Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:48:05Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:05Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:05Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:05Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:48:05Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:05Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:25Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:48:25Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:25Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:48:25Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:25Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:25Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:25Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:25Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:25Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:48:25Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:48:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:48:45Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:48:45Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:48:45Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:48:45Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:45Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:45Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:45Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:45Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:48:45Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:02Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:49:02Z INFO DSPA resources cleaned up. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:14Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:14Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:14Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:14Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:14Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:14Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:14Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:15Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:15Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:15Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:15Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:15Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:15Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:15Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:15Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:15Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:15Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:15Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:15Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:16Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:16Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:16Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:16Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:16Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:16Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:16Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:16Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:16Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:16Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:16Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:17Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:17Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:17Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:17Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:17Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:17Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:17Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:17Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:17Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:17Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:19Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:19Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:19Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:19Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:19Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:19Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:19Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:19Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:19Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:19Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:30Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:30Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:30Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:30Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:30Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:30Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:51Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:51Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:51Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:51Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:49:51Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:49:51Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:51Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:49:51Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:51Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:49:51Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:50:11Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:11Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:50:11Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:50:11Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:11Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:11Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:50:11Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:11Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:11Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:50:31Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:31Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:31Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:31Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:50:31Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:50:31Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:31Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:31Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:31Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:50:51Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:50:51Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:50:51Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:50:51Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:50:51Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:51Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:51Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:51Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:51Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:50:51Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:51:11Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:11Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:11Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:11Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:11Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:11Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:11Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:11Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:11Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:51:31Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:31Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:31Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:31Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:31Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:31Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:31Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:31Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:31Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:51Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:51:51Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:51:51Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:51Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:51Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:51Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:51:51Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:51Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:51Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:51:51Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:52:11Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:11Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:11Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:11Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:11Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:11Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:11Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:11Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:11Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:52:32Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:32Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:32Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:32Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:32Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:32Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:32Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:32Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:32Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:52Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:52:52Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:52:52Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:52Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:52Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:52Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:52Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:52Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:52:52Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:52:52Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:53:12Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:12Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:12Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:12Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:12Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:12Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:12Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:12Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:12Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:53:32Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:32Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:32Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:32Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:32Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:32Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:32Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:32Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:32Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:52Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:53:52Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:53:52Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:52Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:52Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:52Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:52Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:52Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:53:52Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:53:52Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:54:12Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:12Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:54:12Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:12Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:12Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:12Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:12Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:54:12Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:54:12Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:54:32Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:32Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:32Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:32Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:32Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:54:32Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:32Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:54:32Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:54:32Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:52Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:54:52Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:54:52Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:54:52Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:52Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:52Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:52Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:54:52Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:52Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:54:52Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:55:12Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:12Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:12Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:12Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:12Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:12Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:12Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:12Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:12Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:55:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:55:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:55:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:55:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:55:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:55:53Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:55:53Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:53Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:53Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:53Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:53Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:55:53Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:53Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:55:53Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:56:13Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:56:13Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:56:13Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:13Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:13Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:13Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:56:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:56:54Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:56:54Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:54Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:56:54Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:54Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:54Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:54Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:54Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:56:54Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:01Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:57:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:57:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:57:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:57:02Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:57:02Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-08T23:57:02Z INFO DSPA resources cleaned up. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:06Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:06Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:06Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:06Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:06Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:06Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:06Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:06Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:06Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:10Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:10Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:10Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:10Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:10Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:10Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:10Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:10Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:41Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:41Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:41Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:41Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:42Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:58:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:58:42Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:42Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:58:42Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:58:42Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:59:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:02Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:02Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:02Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:02Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:59:22Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:22Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:22Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:22Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:22Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:22Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:22Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:22Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:22Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:59:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:59:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-08T23:59:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-08T23:59:42Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-08T23:59:42Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-08T23:59:42Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:00:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:02Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:02Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:02Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:02Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:00:22Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:22Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:22Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:22Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:22Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:22Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:22Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:22Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:22Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:00:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:00:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:00:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:00:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:01:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:01:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:01:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:01:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:01:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:01:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:01:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:01:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:02:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:02:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:02:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:02:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:02:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:02:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:03:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:03:24Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:24Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:24Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:24Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:24Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:24Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:24Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:24Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:24Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:03:44Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:03:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:03:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:03:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:04:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:04:24Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:24Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:24Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:04:24Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:24Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:24Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:24Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:24Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:04:44Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:04:44Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:04:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:04:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:04:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:05:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:05:25Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:25Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:25Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:25Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:25Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:25Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:25Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:25Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:25Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:05:45Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:05:45Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:05:45Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:05:45Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:05:45Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:05:45Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:05Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:05Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:05Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:05Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:05Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:05Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:05Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:05Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:05Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:05Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:07Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:07Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:07Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:07Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:07Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:07Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:07Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:07Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:07Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:07Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:08Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:08Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:08Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:08Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:08Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:08Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:08Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:08Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:08Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:09Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-09T00:06:09Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-09T00:06:09Z INFO DSPA resources cleaned up. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:09Z INFO DataSciencePipelinesApplication.datasciencepipelinesapplications.opendatahub.io "dspa" not found {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:20Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:31Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:31Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:31Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:31Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:31Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:31Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:31Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:31Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:31Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:32Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:32Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:32Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:32Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:32Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:32Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:32Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:32Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:32Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:06:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:06:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:06:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:06:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:07:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:07:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:07:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:07:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:07:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:07:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:08:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:08:24Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:24Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:24Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:24Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:24Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:24Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:24Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:24Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:24Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:08:44Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:08:44Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:08:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:08:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:08:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:09:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:09:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:09:24Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:24Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:24Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:24Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:24Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:24Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:24Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:24Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:24Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:09:44Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:09:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:09:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:09:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:10:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:24Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:10:25Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:25Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:25Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:25Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:25Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:25Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:25Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:25Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:25Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:10:45Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:10:45Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:45Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:45Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:45Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:45Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:45Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:10:45Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:10:45Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:11:05Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:05Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:05Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:05Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:05Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:05Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:05Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:05Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:05Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:25Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:11:25Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:25Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:25Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:25Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:25Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:25Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:25Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:25Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:25Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:11:45Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:11:45Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:45Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:45Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:45Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:45Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:11:45Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:45Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:11:45Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:12:05Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:05Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:05Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:05Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:05Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:05Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:05Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:05Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:05Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:12:25Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:12:25Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:12:25Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:25Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:25Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:25Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:45Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:46Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:12:46Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:46Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:46Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:12:46Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:46Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:46Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:12:46Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:46Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:46Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:46Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:12:46Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:13:06Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:06Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:06Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:06Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:06Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:06Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:06Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:06Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:06Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:26Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:13:26Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:26Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:26Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:26Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:26Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:26Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:26Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:26Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:26Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:46Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:13:46Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:13:46Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:46Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:46Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:46Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:46Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:46Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:13:46Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:13:46Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:06Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:14:06Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:06Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:06Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:06Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:14:06Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:06Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:06Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:06Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:06Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:14:07Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:07Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:14:07Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:07Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:14:07Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:07Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:07Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-09T00:14:07Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:07Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-09T00:14:07Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-09T00:14:07Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-09T00:14:07Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-09T00:14:07Z INFO DSPA resources cleaned up. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-09T00:14:07Z INFO DataSciencePipelinesApplication.datasciencepipelinesapplications.opendatahub.io "dspa" not found {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} ==================== WORKLOAD POD LOG: rhoai-model-registries/model-catalog-7cc8755dc9-ddw9p.log ==================== W0708 22:57:49.406062 1 connect.go:86] Retrying connection to PostgreSQL (attempt 1/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused W0708 22:57:50.409452 1 connect.go:86] Retrying connection to PostgreSQL (attempt 2/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused W0708 22:57:52.424784 1 connect.go:86] Retrying connection to PostgreSQL (attempt 3/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused W0708 22:57:55.427637 1 connect.go:86] Retrying connection to PostgreSQL (attempt 4/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused W0708 22:57:59.433866 1 connect.go:86] Retrying connection to PostgreSQL (attempt 5/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused 172.30.106.14:5432 (model-catalog-postgres): dial error: dial tcp 172.30.106.14:5432: connect: connection refused I0708 22:58:04.450509 1 connect.go:94] Successfully connected to PostgreSQL database I0708 22:58:04.450558 1 catalog.go:121] Leader election configured: lock duration=1m0s, heartbeat=15s I0708 22:58:04.455758 1 service.go:130] Connecting to EmbedMD service... I0708 22:58:04.455786 1 service.go:137] Connected to EmbedMD service I0708 22:58:04.455791 1 repos.go:35] Getting types... I0708 22:58:04.455832 1 election.go:190] Attempting to acquire leadership... I0708 22:58:04.459363 1 election.go:226] Successfully acquired leadership lock: catalog-leader W0708 22:58:04.467700 1 service.go:186] Unable to get types: no types available: ERROR: relation "Type" does not exist (SQLSTATE 42P01) I0708 22:58:04.468918 1 service.go:162] Running migrations... I0708 22:58:04.682130 1 service.go:167] Migrations completed I0708 22:58:04.682183 1 service.go:169] Syncing types... I0708 22:58:04.746579 1 service.go:174] Syncing types completed W0708 22:58:04.746634 1 election.go:132] Leader callback 0 exited early I0708 22:58:05.468785 1 repos.go:35] Getting types... I0708 22:58:05.469532 1 repos.go:47] Types retrieved I0708 22:58:05.469560 1 repos.go:63] All required types validated successfully 2026/07/08 22:58:05 INFO initializing plugin plugin=mcp version=v1alpha1 basePath=/api/mcp_catalog/v1alpha1 2026/07/08 22:58:05 INFO initializing plugin plugin=model version=v1alpha1 basePath=/api/model_catalog/v1alpha1 I0708 22:58:05.469962 1 performance_metrics.go:154] Loading performance metrics data from [/shared-benchmark-data] I0708 22:58:05.476581 1 performance_metrics.go:239] Built model directory cache (72 models indexed) 2026/07/08 22:58:05 INFO all plugins initialized count=2 2026/07/08 22:58:05 INFO mounting plugin routes plugin=mcp 2026/07/08 22:58:05 INFO mounting plugin routes plugin=model 2026/07/08 22:58:05 INFO starting plugin plugin=mcp I0708 22:58:05.476756 1 base.go:49] Starting mcp plugin in read-only mode (standby) I0708 22:58:05.476772 1 loader.go:84] Initializing MCP loader - parsing configs W0708 22:58:05.477512 1 config.go:24] Configuration file /data/default-sources/sources.yaml uses deprecated 'catalogs' field. Please rename to 'model_catalogs'. I0708 22:58:05.477539 1 loader.go:149] reading MCP catalog config type yaml... I0708 22:58:05.477550 1 loader.go:165] loaded MCP source rh_mcp_servers of type yaml I0708 22:58:05.477554 1 loader.go:149] reading MCP catalog config type yaml... I0708 22:58:05.477558 1 loader.go:165] loaded MCP source rh_partner_mcp_servers of type yaml I0708 22:58:05.477562 1 loader.go:149] reading MCP catalog config type yaml... I0708 22:58:05.477565 1 loader.go:165] loaded MCP source community_mcp_servers of type yaml I0708 22:58:05.477668 1 loader.go:92] MCP loader config parsing complete I0708 22:58:05.477700 1 base.go:64] mcp plugin read-only mode initialized 2026/07/08 22:58:05 INFO starting plugin plugin=model I0708 22:58:05.477715 1 base.go:49] Starting model plugin in read-only mode (standby) W0708 22:58:05.478077 1 config.go:24] Configuration file /data/default-sources/sources.yaml uses deprecated 'catalogs' field. Please rename to 'model_catalogs'. I0708 22:58:05.478102 1 loader.go:204] reading config type yaml... I0708 22:58:05.478116 1 loader.go:222] loaded source redhat_ai_models of type yaml I0708 22:58:05.478120 1 loader.go:204] reading config type yaml... I0708 22:58:05.478124 1 loader.go:222] loaded source redhat_ai_validated_models of type yaml I0708 22:58:05.478127 1 loader.go:204] reading config type yaml... I0708 22:58:05.478131 1 loader.go:222] loaded source other_models of type yaml I0708 22:58:05.478266 1 base.go:64] model plugin read-only mode initialized I0708 22:58:05.478764 1 catalog.go:176] Catalog API server listening on 0.0.0.0:8080 I0708 22:58:05.478781 1 catalog.go:195] Became leader - notifying plugins 2026/07/08 22:58:05 INFO plugin becoming leader plugin=mcp 2026/07/08 22:58:05 INFO plugin becoming leader plugin=model I0708 22:58:05.478889 1 base.go:97] mcp plugin becoming leader I0708 22:58:05.478910 1 base.go:97] model plugin becoming leader I0708 22:58:05.478988 1 loader.go:101] MCP loader performing leader operations I0708 22:58:05.479012 1 loader.go:192] Loading MCP servers from source: Red Hat MCP Servers (id: rh_mcp_servers) I0708 22:58:05.485385 1 loader.go:379] Reading models from yaml source redhat_ai_models I0708 22:58:05.497373 1 loader.go:379] Reading models from yaml source redhat_ai_validated_models I0708 22:58:05.596001 1 loader.go:379] Reading models from yaml source other_models I0708 22:58:05.601223 1 base.go:112] model plugin leader mode active I0708 22:58:05.601252 1 loader.go:285] Loading model redhat_ai_models:RedHatAI/granite-3.3-8b-instruct with 1 artifact(s) I0708 22:58:05.616599 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Apertus-8B-Instruct-2509-FP8-dynamic with 1 artifact(s) I0708 22:58:05.664472 1 loader.go:192] Loading MCP servers from source: Red Hat Partner MCP Servers (id: rh_partner_mcp_servers) I0708 22:58:05.852446 1 performance_metrics.go:273] Loaded 117 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Apertus-8B-Instruct-2509-FP8-dynamic I0708 22:58:05.852605 1 loader.go:285] Loading model other_models:RedHatAI/Qwen3-Embedding-8B with 1 artifact(s) I0708 22:58:05.863012 1 loader.go:285] Loading model redhat_ai_models:RedHatAI/granite-4.0-h-small with 1 artifact(s) I0708 22:58:05.870089 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/DeepSeek-R1-0528-quantized.w4a16 with 1 artifact(s) I0708 22:58:06.047429 1 performance_metrics.go:273] Loaded 55 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/DeepSeek-R1-0528-quantized.w4a16 I0708 22:58:06.047488 1 loader.go:285] Loading model other_models:RedHatAI/all-MiniLM-L6-v2 with 1 artifact(s) I0708 22:58:06.057403 1 loader.go:285] Loading model redhat_ai_models:RedHatAI/granite-4.0-h-tiny with 1 artifact(s) I0708 22:58:06.066185 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Devstral-Small-2-24B-Instruct-2512 with 1 artifact(s) I0708 22:58:06.131921 1 loader.go:192] Loading MCP servers from source: Community MCP Servers (id: community_mcp_servers) I0708 22:58:06.241232 1 loader.go:115] MCP loader leader operations complete I0708 22:58:06.241281 1 base.go:112] mcp plugin leader mode active I0708 22:58:06.383455 1 performance_metrics.go:273] Loaded 138 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Devstral-Small-2-24B-Instruct-2512 I0708 22:58:06.383512 1 loader.go:285] Loading model other_models:RedHatAI/embeddinggemma-300m with 1 artifact(s) I0708 22:58:06.392090 1 loader.go:285] Loading model redhat_ai_models:granite-7b-redhat-lab with 1 artifact(s) I0708 22:58:06.431835 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Kimi-K2-Instruct-quantized.w4a16 with 1 artifact(s) I0708 22:58:06.474939 1 performance_metrics.go:273] Loaded 22 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Kimi-K2-Instruct-quantized.w4a16 I0708 22:58:06.474995 1 loader.go:285] Loading model other_models:RedHatAI/granite-embedding-english-r2 with 1 artifact(s) I0708 22:58:06.484269 1 loader.go:285] Loading model redhat_ai_models:granite-8b-starter-v1 with 1 artifact(s) I0708 22:58:06.493054 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.1-8B-Instruct with 1 artifact(s) I0708 22:58:06.737853 1 performance_metrics.go:273] Loaded 107 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.1-8B-Instruct I0708 22:58:06.737961 1 loader.go:285] Loading model other_models:RedHatAI/nomic-embed-text-v1.5 with 1 artifact(s) I0708 22:58:06.738073 1 loader.go:420] other_models: loaded 5 models I0708 22:58:06.753547 1 loader.go:431] other_models: cleaned up 0 models I0708 22:58:06.754275 1 loader.go:285] Loading model redhat_ai_models:granite-8b-lab-v1 with 1 artifact(s) I0708 22:58:06.766860 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF with 1 artifact(s) I0708 22:58:06.869555 1 performance_metrics.go:273] Loaded 30 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF I0708 22:58:06.869620 1 loader.go:285] Loading model redhat_ai_models:granite-3.1-8b-lab-v1 with 1 artifact(s) I0708 22:58:06.878203 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-dynamic with 1 artifact(s) I0708 22:58:06.950188 1 performance_metrics.go:273] Loaded 35 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-dynamic I0708 22:58:06.950257 1 loader.go:285] Loading model redhat_ai_models:granite-8b-code-instruct with 1 artifact(s) I0708 22:58:06.957458 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct with 1 artifact(s) I0708 22:58:07.038758 1 performance_metrics.go:273] Loaded 35 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct I0708 22:58:07.038828 1 loader.go:285] Loading model redhat_ai_models:granite-8b-code-base with 1 artifact(s) I0708 22:58:07.049785 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-FP8-dynamic with 1 artifact(s) I0708 22:58:07.138139 1 performance_metrics.go:273] Loaded 36 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-FP8-dynamic I0708 22:58:07.138220 1 loader.go:285] Loading model redhat_ai_models:prometheus-8x7b-v2-0 with 1 artifact(s) I0708 22:58:07.138320 1 loader.go:420] redhat_ai_models: loaded 10 models I0708 22:58:07.154744 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-quantized.w4a16 with 1 artifact(s) I0708 22:58:07.155296 1 loader.go:431] redhat_ai_models: cleaned up 0 models I0708 22:58:07.248770 1 performance_metrics.go:273] Loaded 30 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-quantized.w4a16 I0708 22:58:07.248843 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-quantized.w8a8 with 1 artifact(s) I0708 22:58:07.342121 1 performance_metrics.go:273] Loaded 43 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-quantized.w8a8 I0708 22:58:07.342203 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Maverick-17B-128E-Instruct with 1 artifact(s) I0708 22:58:07.372864 1 performance_metrics.go:273] Loaded 3 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Maverick-17B-128E-Instruct I0708 22:58:07.372916 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Maverick-17B-128E-Instruct-FP8 with 1 artifact(s) I0708 22:58:07.432585 1 performance_metrics.go:273] Loaded 18 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Maverick-17B-128E-Instruct-FP8 I0708 22:58:07.432727 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct with 1 artifact(s) I0708 22:58:07.477097 1 performance_metrics.go:273] Loaded 15 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct I0708 22:58:07.477169 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct-FP8-dynamic with 1 artifact(s) I0708 22:58:07.545527 1 performance_metrics.go:273] Loaded 40 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct-FP8-dynamic I0708 22:58:07.545665 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct-quantized.w4a16 with 1 artifact(s) I0708 22:58:07.584439 1 performance_metrics.go:273] Loaded 16 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct-quantized.w4a16 I0708 22:58:07.584564 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic with 1 artifact(s) I0708 22:58:07.630855 1 performance_metrics.go:273] Loaded 3 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic I0708 22:58:07.630919 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w4a16 with 1 artifact(s) I0708 22:58:07.645045 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8 with 1 artifact(s) I0708 22:58:07.657773 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/MiniMax-M2.5 with 1 artifact(s) I0708 22:58:07.733757 1 performance_metrics.go:273] Loaded 41 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/MiniMax-M2.5 I0708 22:58:07.733878 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Ministral-3-14B-Instruct-2512 with 1 artifact(s) I0708 22:58:07.935701 1 performance_metrics.go:273] Loaded 130 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Ministral-3-14B-Instruct-2512 I0708 22:58:07.935756 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Ministral-3-3B-Instruct-2512 with 1 artifact(s) I0708 22:58:08.159381 1 performance_metrics.go:273] Loaded 149 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Ministral-3-3B-Instruct-2512 I0708 22:58:08.159447 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Large-3-675B-Instruct-2512 with 1 artifact(s) I0708 22:58:08.221283 1 performance_metrics.go:273] Loaded 18 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Large-3-675B-Instruct-2512 I0708 22:58:08.221350 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Large-3-675B-Instruct-2512-NVFP4 with 1 artifact(s) I0708 22:58:08.282136 1 performance_metrics.go:273] Loaded 42 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Large-3-675B-Instruct-2512-NVFP4 I0708 22:58:08.282186 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501 with 1 artifact(s) I0708 22:58:08.431451 1 performance_metrics.go:273] Loaded 81 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501 I0708 22:58:08.431520 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501-FP8-dynamic with 1 artifact(s) I0708 22:58:08.442300 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501-quantized.w4a16 with 1 artifact(s) I0708 22:58:08.453694 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501-quantized.w8a8 with 1 artifact(s) I0708 22:58:08.463742 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503 with 1 artifact(s) I0708 22:58:08.640351 1 performance_metrics.go:273] Loaded 105 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503 I0708 22:58:08.640417 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-FP8-dynamic with 1 artifact(s) I0708 22:58:08.841072 1 performance_metrics.go:273] Loaded 112 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-FP8-dynamic I0708 22:58:08.841174 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-quantized.w4a16 with 1 artifact(s) I0708 22:58:08.987042 1 performance_metrics.go:273] Loaded 104 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-quantized.w4a16 I0708 22:58:08.987190 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-quantized.w8a8 with 1 artifact(s) I0708 22:58:09.185856 1 performance_metrics.go:273] Loaded 123 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-quantized.w8a8 I0708 22:58:09.185961 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-4-119B-2603 with 1 artifact(s) I0708 22:58:09.277299 1 performance_metrics.go:273] Loaded 40 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-4-119B-2603 I0708 22:58:09.277354 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-4-119B-2603-NVFP4 with 1 artifact(s) I0708 22:58:09.439796 1 performance_metrics.go:273] Loaded 91 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-4-119B-2603-NVFP4 I0708 22:58:09.439913 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mixtral-8x7B-Instruct-v0.1 with 1 artifact(s) I0708 22:58:09.552451 1 performance_metrics.go:273] Loaded 59 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mixtral-8x7B-Instruct-v0.1 I0708 22:58:09.552503 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 with 1 artifact(s) I0708 22:58:09.747935 1 performance_metrics.go:273] Loaded 126 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 I0708 22:58:09.748007 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 with 1 artifact(s) I0708 22:58:09.879580 1 performance_metrics.go:273] Loaded 80 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 I0708 22:58:09.879639 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 with 1 artifact(s) I0708 22:58:10.034673 1 performance_metrics.go:273] Loaded 79 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 I0708 22:58:10.034772 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 with 1 artifact(s) I0708 22:58:10.094385 1 performance_metrics.go:273] Loaded 31 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 I0708 22:58:10.094447 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamic with 1 artifact(s) I0708 22:58:10.189240 1 performance_metrics.go:273] Loaded 46 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamic I0708 22:58:10.189370 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Phi-4-mini-instruct-FP8-dynamic with 1 artifact(s) I0708 22:58:10.368471 1 performance_metrics.go:273] Loaded 110 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Phi-4-mini-instruct-FP8-dynamic I0708 22:58:10.368622 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Phi-4-reasoning-FP8-dynamic with 1 artifact(s) I0708 22:58:10.547799 1 performance_metrics.go:273] Loaded 75 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Phi-4-reasoning-FP8-dynamic I0708 22:58:10.548060 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct with 1 artifact(s) I0708 22:58:10.741159 1 performance_metrics.go:273] Loaded 102 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct I0708 22:58:10.741233 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-FP8-dynamic with 1 artifact(s) I0708 22:58:10.935662 1 performance_metrics.go:273] Loaded 110 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-FP8-dynamic I0708 22:58:10.935731 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-quantized.w4a16 with 1 artifact(s) I0708 22:58:11.069404 1 performance_metrics.go:273] Loaded 87 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-quantized.w4a16 I0708 22:58:11.069458 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8 with 1 artifact(s) I0708 22:58:11.257489 1 performance_metrics.go:273] Loaded 106 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8 I0708 22:58:11.257555 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-8B-FP8-dynamic with 1 artifact(s) I0708 22:58:11.486588 1 performance_metrics.go:273] Loaded 155 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-8B-FP8-dynamic I0708 22:58:11.486637 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-Coder-480B-A35B-Instruct-FP8 with 1 artifact(s) I0708 22:58:11.565445 1 performance_metrics.go:273] Loaded 13 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-Coder-480B-A35B-Instruct-FP8 I0708 22:58:11.565489 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-Coder-Next-NVFP4 with 1 artifact(s) I0708 22:58:11.755669 1 performance_metrics.go:273] Loaded 119 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-Coder-Next-NVFP4 I0708 22:58:11.755767 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-Next-80B-A3B-Instruct-quantized.w4a16 with 1 artifact(s) I0708 22:58:11.961289 1 performance_metrics.go:273] Loaded 133 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-Next-80B-A3B-Instruct-quantized.w4a16 I0708 22:58:11.961346 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-VL-235B-A22B-Instruct-NVFP4 with 1 artifact(s) I0708 22:58:12.127063 1 performance_metrics.go:273] Loaded 79 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-VL-235B-A22B-Instruct-NVFP4 I0708 22:58:12.127159 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3.5-122B-A10B-FP8-dynamic with 1 artifact(s) I0708 22:58:12.193858 1 performance_metrics.go:273] Loaded 42 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3.5-122B-A10B-FP8-dynamic I0708 22:58:12.193942 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3.5-35B-A3B-FP8-dynamic with 1 artifact(s) I0708 22:58:12.378952 1 performance_metrics.go:273] Loaded 108 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3.5-35B-A3B-FP8-dynamic I0708 22:58:12.379022 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3.5-397B-A17B-FP8-dynamic with 1 artifact(s) I0708 22:58:12.468939 1 performance_metrics.go:273] Loaded 25 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3.5-397B-A17B-FP8-dynamic I0708 22:58:12.468988 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Voxtral-Mini-3B-2507-FP8-dynamic with 1 artifact(s) I0708 22:58:12.482610 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gemma-2-9b-it with 1 artifact(s) I0708 22:58:12.494613 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gemma-2-9b-it-FP8 with 1 artifact(s) I0708 22:58:12.532460 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gemma-3n-E4B-it-FP8-dynamic with 1 artifact(s) I0708 22:58:12.660427 1 performance_metrics.go:273] Loaded 81 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gemma-3n-E4B-it-FP8-dynamic I0708 22:58:12.660475 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gpt-oss-120b with 1 artifact(s) I0708 22:58:12.881298 1 performance_metrics.go:273] Loaded 148 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gpt-oss-120b I0708 22:58:12.881355 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gpt-oss-120b-essential with 1 artifact(s) I0708 22:58:13.146119 1 performance_metrics.go:273] Loaded 148 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gpt-oss-120b-essential I0708 22:58:13.146168 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gpt-oss-20b with 1 artifact(s) I0708 22:58:13.435820 1 performance_metrics.go:273] Loaded 177 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gpt-oss-20b I0708 22:58:13.435876 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gpt-oss-20b-essential with 1 artifact(s) I0708 22:58:13.674080 1 performance_metrics.go:273] Loaded 177 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gpt-oss-20b-essential I0708 22:58:13.674330 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-base-quantized.w4a16 with 1 artifact(s) I0708 22:58:13.728130 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct with 2 artifact(s) I0708 22:58:13.873247 1 performance_metrics.go:273] Loaded 95 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct I0708 22:58:13.873293 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-FP8-dynamic with 1 artifact(s) I0708 22:58:13.939318 1 performance_metrics.go:273] Loaded 6 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-FP8-dynamic I0708 22:58:13.939358 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-quantized.w4a16 with 1 artifact(s) I0708 22:58:13.967678 1 performance_metrics.go:273] Loaded 4 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-quantized.w4a16 I0708 22:58:13.967732 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-quantized.w8a8 with 1 artifact(s) I0708 22:58:13.994979 1 performance_metrics.go:273] Loaded 1 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-quantized.w8a8 I0708 22:58:13.995016 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-4.0-h-small-FP8-dynamic with 1 artifact(s) I0708 22:58:14.086851 1 performance_metrics.go:273] Loaded 42 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-4.0-h-small-FP8-dynamic I0708 22:58:14.086931 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-4.0-h-tiny-FP8-dynamic with 1 artifact(s) I0708 22:58:14.193189 1 performance_metrics.go:273] Loaded 46 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-4.0-h-tiny-FP8-dynamic I0708 22:58:14.193243 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/phi-4 with 1 artifact(s) I0708 22:58:14.332623 1 performance_metrics.go:273] Loaded 55 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/phi-4 I0708 22:58:14.332676 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/phi-4-FP8-dynamic with 1 artifact(s) I0708 22:58:14.465296 1 performance_metrics.go:273] Loaded 54 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/phi-4-FP8-dynamic I0708 22:58:14.465366 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/phi-4-quantized.w4a16 with 1 artifact(s) I0708 22:58:14.571648 1 performance_metrics.go:273] Loaded 52 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/phi-4-quantized.w4a16 I0708 22:58:14.571710 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/phi-4-quantized.w8a8 with 1 artifact(s) I0708 22:58:14.727459 1 performance_metrics.go:273] Loaded 60 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/phi-4-quantized.w8a8 I0708 22:58:14.727528 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/sarvam-105b-FP8-dynamic with 1 artifact(s) I0708 22:58:14.840792 1 performance_metrics.go:273] Loaded 61 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/sarvam-105b-FP8-dynamic I0708 22:58:14.840844 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/sarvam-30b-FP8-dynamic with 1 artifact(s) I0708 22:58:15.031365 1 performance_metrics.go:273] Loaded 105 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/sarvam-30b-FP8-dynamic I0708 22:58:15.031441 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/whisper-large-v3-turbo-quantized.w4a16 with 1 artifact(s) I0708 22:58:15.031531 1 loader.go:420] redhat_ai_validated_models: loaded 76 models I0708 22:58:15.072446 1 loader.go:431] redhat_ai_validated_models: cleaned up 0 models 2026/07/08 23:11:44 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:37304 - 200 472B in 560.489µs 2026/07/08 23:11:44 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:37318 - 200 420B in 3.117669ms 2026/07/08 23:11:44 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:37312 - 200 9130B in 5.145067ms 2026/07/08 23:11:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:37318 - 200 141549B in 8.779821ms 2026/07/08 23:11:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:37304 - 200 47545B in 3.776859ms 2026/07/08 23:11:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:37304 - 200 84064B in 3.996625ms 2026/07/08 23:11:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37346 - 200 56B in 41.021488ms 2026/07/08 23:11:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37318 - 200 41125B in 52.955106ms 2026/07/08 23:11:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37304 - 200 20726B in 53.391881ms 2026/07/08 23:11:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37342 - 200 62309B in 78.714817ms 2026/07/08 23:11:59 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic HTTP/1.1" from 127.0.0.1:37318 - 200 16070B in 2.480827ms 2026/07/08 23:11:59 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts HTTP/1.1" from 127.0.0.1:37346 - 200 32975B in 7.996924ms 2026/07/08 23:11:59 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts HTTP/1.1" from 127.0.0.1:37346 - 200 32975B in 7.563976ms 2026/07/08 23:11:59 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?filterQuery=variant_group_id.string_value%3D%22c359df52-82cb-4d43-be32-6dafea427522%22&pageSize=10&source=redhat_ai_validated_models HTTP/1.1" from 127.0.0.1:37346 - 200 16124B in 3.63302ms 2026/07/08 23:12:00 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90&pageSize=20&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37346 - 200 41124B in 25.591849ms 2026/07/08 23:12:00 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&pageSize=20&recommendations=true&targetRPS=1 HTTP/1.1" from 127.0.0.1:37346 - 200 41124B in 22.843553ms 2026/07/08 23:12:36 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:37318 - 200 472B in 211.478µs 2026/07/08 23:12:36 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:37346 - 200 420B in 3.113802ms 2026/07/08 23:12:36 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:57354 - 200 9130B in 3.863389ms 2026/07/08 23:12:36 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:37346 - 200 141549B in 7.028486ms 2026/07/08 23:12:36 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:37346 - 200 47545B in 3.639854ms 2026/07/08 23:12:36 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:37318 - 200 84064B in 5.258591ms 2026/07/08 23:12:37 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37346 - 200 20726B in 28.28212ms 2026/07/08 23:12:37 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37318 - 200 41125B in 32.775457ms 2026/07/08 23:12:37 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37318 - 200 56B in 4.267338ms 2026/07/08 23:12:37 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37346 - 200 62309B in 33.388621ms 2026/07/08 23:12:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic HTTP/1.1" from 127.0.0.1:37318 - 200 16070B in 1.8785ms 2026/07/08 23:12:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts HTTP/1.1" from 127.0.0.1:37346 - 200 32975B in 6.322134ms 2026/07/08 23:12:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts HTTP/1.1" from 127.0.0.1:37346 - 200 32975B in 6.846204ms 2026/07/08 23:12:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?filterQuery=variant_group_id.string_value%3D%22c359df52-82cb-4d43-be32-6dafea427522%22&pageSize=10&source=redhat_ai_validated_models HTTP/1.1" from 127.0.0.1:37346 - 200 16124B in 2.020408ms 2026/07/08 23:12:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&pageSize=20&recommendations=true&targetRPS=1 HTTP/1.1" from 127.0.0.1:37346 - 200 41124B in 18.357415ms 2026/07/08 23:12:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90&pageSize=20&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:37346 - 200 41124B in 23.999485ms 2026/07/08 23:16:15 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36792 - 200 420B in 1.178024ms 2026/07/08 23:16:21 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36792 - 200 420B in 1.453069ms 2026/07/08 23:16:38 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36792 - 200 420B in 1.091386ms 2026/07/08 23:17:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:38096 - 200 472B in 193.795µs 2026/07/08 23:17:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:38096 - 200 9130B in 1.763914ms 2026/07/08 23:17:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36792 - 200 420B in 2.789825ms 2026/07/08 23:17:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:36792 - 200 47545B in 3.826351ms 2026/07/08 23:17:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:38096 - 200 141549B in 5.818365ms 2026/07/08 23:17:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:36792 - 200 84064B in 3.743249ms 2026/07/08 23:17:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:38096 - 200 56B in 8.055833ms 2026/07/08 23:17:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36792 - 200 20726B in 103.851131ms 2026/07/08 23:17:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:38114 - 200 41125B in 122.550707ms 2026/07/08 23:17:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:38124 - 200 62309B in 127.205127ms 2026/07/08 23:17:19 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:36792 - 200 472B in 547.961µs 2026/07/08 23:17:19 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:42222 - 200 420B in 1.555584ms 2026/07/08 23:17:19 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:38096 - 200 9130B in 3.565584ms 2026/07/08 23:17:19 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:42222 - 200 47545B in 4.148925ms 2026/07/08 23:17:19 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:36792 - 200 141549B in 7.707637ms 2026/07/08 23:17:19 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:36792 - 200 84064B in 4.904276ms 2026/07/08 23:17:20 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42246 - 200 56B in 20.556373ms 2026/07/08 23:17:20 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42222 - 200 20726B in 28.497104ms 2026/07/08 23:17:20 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36792 - 200 62309B in 116.195317ms 2026/07/08 23:17:20 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42262 - 200 41125B in 125.187645ms 2026/07/08 23:17:49 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:42222 - 200 472B in 104.444µs 2026/07/08 23:17:49 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:42246 - 200 420B in 1.666566ms 2026/07/08 23:17:49 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:42782 - 200 9130B in 3.661912ms 2026/07/08 23:17:49 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:42246 - 200 84064B in 7.299896ms 2026/07/08 23:17:49 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:42222 - 200 141549B in 7.707849ms 2026/07/08 23:17:49 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:42802 - 200 47545B in 18.135376ms 2026/07/08 23:17:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42826 - 200 56B in 21.79108ms 2026/07/08 23:17:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42246 - 200 41125B in 117.763323ms 2026/07/08 23:17:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42222 - 200 62309B in 123.32625ms 2026/07/08 23:17:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42814 - 200 20726B in 125.706915ms 2026/07/08 23:18:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?filterQuery=artifacts.use_case.string_value%3D%27chatbot%27+AND+artifacts.ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=artifacts.ttft_p90.double_value&pageSize=10&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42246 - 200 292781B in 1.292742981s 2026/07/08 23:18:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FMinistral-3-3B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42246 - 200 29027B in 40.936998ms 2026/07/08 23:18:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FMinistral-3-14B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42246 - 200 59712B in 31.522723ms 2026/07/08 23:18:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49208 - 200 41125B in 232.493635ms 2026/07/08 23:18:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49204 - 200 62309B in 326.360712ms 2026/07/08 23:18:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fsarvam-30b-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49184 - 200 24427B in 331.422668ms 2026/07/08 23:18:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FQwen2.5-7B-Instruct-quantized.w8a8/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42246 - 200 35997B in 250.503018ms 2026/07/08 23:18:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgpt-oss-20b/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49192 - 200 52497B in 416.795153ms 2026/07/08 23:18:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgpt-oss-20b-essential/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:42826 - 200 52679B in 427.646236ms 2026/07/08 23:18:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FQwen3-8B-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49204 - 200 37135B in 39.187107ms 2026/07/08 23:18:04 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FLlama-3.1-8B-Instruct/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49208 - 200 55242B in 38.755897ms 2026/07/08 23:18:24 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:49204 - 200 472B in 80.886µs 2026/07/08 23:18:24 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:49208 - 200 420B in 1.44345ms 2026/07/08 23:18:24 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:49208 - 200 9130B in 3.318437ms 2026/07/08 23:18:25 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:49208 - 200 141549B in 7.132478ms 2026/07/08 23:18:25 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:49204 - 200 84064B in 8.106402ms 2026/07/08 23:18:25 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:49204 - 200 47545B in 3.842707ms 2026/07/08 23:18:25 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49208 - 200 20726B in 28.569146ms 2026/07/08 23:18:25 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49204 - 200 62309B in 114.725979ms 2026/07/08 23:18:25 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:57888 - 200 56B in 105.831585ms 2026/07/08 23:18:25 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:57876 - 200 41125B in 125.727939ms 2026/07/08 23:18:38 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?filterQuery=artifacts.use_case.string_value%3D%27chatbot%27+AND+artifacts.ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=artifacts.ttft_p90.double_value&pageSize=10&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49204 - 200 292781B in 1.242871298s 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FMinistral-3-14B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49204 - 200 59712B in 47.261137ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgpt-oss-20b/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49208 - 200 52497B in 124.250568ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:59760 - 200 41125B in 207.780878ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FMinistral-3-3B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49204 - 200 29027B in 377.80654ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fsarvam-30b-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:59784 - 200 24427B in 342.747005ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgpt-oss-20b-essential/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:59800 - 200 52679B in 425.423092ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:59776 - 200 62309B in 483.477412ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FQwen2.5-7B-Instruct-quantized.w8a8/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49208 - 200 35997B in 347.977873ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FQwen3-8B-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:59760 - 200 37135B in 334.392151ms 2026/07/08 23:18:39 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FLlama-3.1-8B-Instruct/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:59784 - 200 55242B in 170.659635ms 2026/07/08 23:18:52 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:59800 - 200 420B in 2.567001ms 2026/07/08 23:18:52 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:49204 - 200 472B in 101.852µs 2026/07/08 23:18:52 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:49204 - 200 9130B in 3.543938ms 2026/07/08 23:18:53 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:49204 - 200 47545B in 10.10661ms 2026/07/08 23:18:53 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:59800 - 200 84064B in 11.239226ms 2026/07/08 23:18:53 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:34514 - 200 141549B in 21.819886ms 2026/07/08 23:18:53 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:59800 - 200 56B in 7.205139ms 2026/07/08 23:18:53 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:49204 - 200 62309B in 123.834411ms 2026/07/08 23:18:53 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:34544 - 200 20726B in 123.746819ms 2026/07/08 23:18:53 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:34530 - 200 41125B in 134.71244ms W0708 22:57:49.500847 1 deprecated.go:66] ==== Removed Flag Warning ====================== logtostderr is removed in the k8s upstream and has no effect any more. =============================================== I0708 22:57:49.500987 1 kube-rbac-proxy.go:553] Reading config file: /etc/kube-rbac-proxy/config-file.yaml I0708 22:57:49.502155 1 kube-rbac-proxy.go:247] Valid token audiences: I0708 22:57:49.503069 1 kube-rbac-proxy.go:370] Reading certificate files I0708 22:57:49.503577 1 kube-rbac-proxy.go:418] Starting TCP socket on 0.0.0.0:8443 I0708 22:57:49.503629 1 kube-rbac-proxy.go:466] Starting TCP socket on 0.0.0.0:8888 I0708 22:57:49.503990 1 kube-rbac-proxy.go:425] Listening securely on 0.0.0.0:8443 I0708 22:57:49.504002 1 kube-rbac-proxy.go:473] Listening securely on 0.0.0.0:8888 for proxy endpoints ==================== WORKLOAD POD LOG: rhoai-model-registries/model-catalog-postgres-686d85b578-pl9ls.log ==================== The files belonging to this database system will be owned by user "postgres". This user must also own the server process. The database cluster will be initialized with locale "en_US.utf8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english". Data page checksums are disabled. fixing permissions on existing directory /var/lib/pgsql/data/userdata ... ok creating subdirectories ... ok selecting dynamic shared memory implementation ... posix selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting default time zone ... UTC creating configuration files ... ok running bootstrap script ... ok performing post-bootstrap initialization ... ok syncing data to disk ... ok Success. You can now start the database server using: pg_ctl -D /var/lib/pgsql/data/userdata -l logfile start initdb: warning: enabling "trust" authentication for local connections initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. waiting for server to start....2026-07-08 22:57:54.740 UTC [30] LOG: redirecting log output to logging collector process 2026-07-08 22:57:54.740 UTC [30] HINT: Future log output will appear in directory "log". done server started /var/run/postgresql:5432 - accepting connections => sourcing /usr/share/container-scripts/postgresql/start/set_passwords.sh ... ALTER ROLE waiting for server to shut down.... done server stopped Starting server... 2026-07-08 22:57:55.132 UTC [1] LOG: redirecting log output to logging collector process 2026-07-08 22:57:55.132 UTC [1] HINT: Future log output will appear in directory "log".