go test ./test/e2e/upgrade/post/ -v -timeout 10m Starting post-upgrade smoke test suite === RUN TestPostUpgradeControllerRunning --- PASS: TestPostUpgradeControllerRunning (0.00s) === RUN TestPostUpgradeTrainerStillReady post_upgrade_test.go:81: Timed out after 300.000s. The function passed to Eventually failed at /workspace/source/src_code/test/e2e/upgrade/post/post_upgrade_test.go:64 with: Expected : Not Ready to equal : Ready client.go:120: Controller logs: 2026-09-05T03:28:37Z INFO setup starting manager 2026-09-05T03:28:37Z INFO controller-runtime.metrics Starting metrics server 2026-09-05T03:28:37Z INFO starting server {"name": "health probe", "addr": "[::]:8081"} 2026-09-05T03:28:37Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8443", "secure": true} 2026-09-05T03:28:37Z INFO Starting EventSource {"controller": "tlssecurityprofilewatcher", "source": "kind source: *v1.APIServer"} 2026-09-05T03:28:37Z INFO Starting EventSource {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "source": "kind source: *v1.ConfigMap"} 2026-09-05T03:28:37Z INFO Starting EventSource {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "source": "kind source: *unstructured.Unstructured"} 2026-09-05T03:28:37Z INFO Starting EventSource {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "source": "kind source: *unstructured.Unstructured"} 2026-09-05T03:28:37Z INFO Starting EventSource {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "source": "kind source: *unstructured.Unstructured"} 2026-09-05T03:28:37Z INFO Starting EventSource {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "source": "kind source: *unstructured.Unstructured"} 2026-09-05T03:28:37Z INFO Starting EventSource {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "source": "kind source: *unstructured.Unstructured"} 2026-09-05T03:28:38Z INFO Starting Controller {"controller": "tlssecurityprofilewatcher"} 2026-09-05T03:28:38Z INFO Starting workers {"controller": "tlssecurityprofilewatcher", "worker count": 1} 2026-09-05T03:28:38Z INFO Starting Controller {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer"} 2026-09-05T03:28:38Z INFO Starting workers {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "worker count": 1} 2026-09-05T03:28:38Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06"} 2026-09-05T03:28:38Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06"} 2026-09-05T03:28:38Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:38Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:38Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:38Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:39 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:39Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "count": 21} 2026/09/05 03:28:39 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:39Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "count": 14} 2026/09/05 03:28:39 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:39Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "count": 3} 2026-09-05T03:28:39Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:40Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99b99c71-58c9-4c18-84d8-4ff694615a06", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:40Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4"} 2026-09-05T03:28:40Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4"} 2026-09-05T03:28:40Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:40Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:40Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:40Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:41 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:41Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "count": 21} 2026/09/05 03:28:41 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:41Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "count": 14} 2026/09/05 03:28:41 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:41Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "count": 3} 2026-09-05T03:28:41Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:41Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "0cb639c2-df06-49d0-a3ec-e9c496d684c4", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:41Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5"} 2026-09-05T03:28:41Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5"} 2026-09-05T03:28:41Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:41Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:41Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:41Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:42 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:42Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "count": 21} 2026/09/05 03:28:42 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:42Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "count": 14} 2026/09/05 03:28:42 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:42Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "count": 3} 2026-09-05T03:28:42Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:42Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3d07eb37-9e8d-4358-a555-c69521d95db5", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:42Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34"} 2026-09-05T03:28:42Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34"} 2026-09-05T03:28:42Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:43Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:43Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:43Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:43 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:43Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "count": 21} 2026/09/05 03:28:43 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:43Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "count": 14} 2026/09/05 03:28:43 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:43Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "count": 3} 2026-09-05T03:28:43Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:44Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "8cc2403c-5b4b-420c-b8b4-d146f8d32b34", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:44Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da"} 2026-09-05T03:28:44Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da"} 2026-09-05T03:28:44Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:44Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:44Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:44Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:44 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:45Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "count": 21} 2026/09/05 03:28:45 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:45Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "count": 14} 2026/09/05 03:28:45 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:45Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "count": 3} 2026-09-05T03:28:45Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:45Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "76b76e5a-d680-4f07-aa9c-20fc295163da", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:45Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc"} 2026-09-05T03:28:45Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc"} 2026-09-05T03:28:45Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:45Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:45Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:45Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:46 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:46Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "count": 21} 2026/09/05 03:28:46 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:46Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "count": 14} 2026/09/05 03:28:46 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:46Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "count": 3} 2026-09-05T03:28:46Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:46Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "7bb55fe0-5b70-413b-a4b2-03d538e73efc", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:46Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57"} 2026-09-05T03:28:46Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57"} 2026-09-05T03:28:46Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:46Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:46Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:46Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:47 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:47Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "count": 21} 2026/09/05 03:28:47 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:47Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "count": 14} 2026/09/05 03:28:47 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:47Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "count": 3} 2026-09-05T03:28:47Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:47Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "44243bfe-3bf9-41ec-b062-ce7ffa8b8c57", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:48Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0"} 2026-09-05T03:28:48Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0"} 2026-09-05T03:28:48Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:48Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:48Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:48Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:48 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:48Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "count": 21} 2026/09/05 03:28:48 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:48Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "count": 14} 2026/09/05 03:28:48 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:48Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "count": 3} 2026-09-05T03:28:48Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:49Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "a5e792cf-0e68-40f0-adb5-c3bbc28ff2f0", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:49Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4"} 2026-09-05T03:28:49Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4"} 2026-09-05T03:28:49Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:49Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:49Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:49Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:50 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:50Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "count": 21} 2026/09/05 03:28:50 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:50Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "count": 14} 2026/09/05 03:28:50 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:50Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "count": 3} 2026-09-05T03:28:50Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:51Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "28c627fb-d8ad-4c53-acb1-20d8b1732cd4", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:52Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a"} 2026-09-05T03:28:52Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a"} 2026-09-05T03:28:52Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:52Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:52Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:52Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:53 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:53Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "count": 21} 2026/09/05 03:28:53 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:53Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "count": 14} 2026/09/05 03:28:53 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:53Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "count": 3} 2026-09-05T03:28:53Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:53Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "99499aba-90bc-418e-b95d-d26ebb4ed99a", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:28:56Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7"} 2026-09-05T03:28:56Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7"} 2026-09-05T03:28:56Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:28:56Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:28:56Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:28:56Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:28:56 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:57Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "count": 21} 2026/09/05 03:28:57 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:57Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "count": 14} 2026/09/05 03:28:57 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:28:57Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "count": 3} 2026-09-05T03:28:57Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:28:57Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "934fa59b-5836-4219-b775-9b44fd99c5d7", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:29:02Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7"} 2026-09-05T03:29:02Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7"} 2026-09-05T03:29:02Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:29:02Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:29:02Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:29:02Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:29:03 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:03Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "count": 21} 2026/09/05 03:29:03 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:03Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "count": 14} 2026/09/05 03:29:03 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:03Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "count": 3} 2026-09-05T03:29:03Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:29:03Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "3bf3c87d-4132-4ca2-880e-a3873b73e0e7", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:29:13Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0"} 2026-09-05T03:29:13Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0"} 2026-09-05T03:29:13Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:29:13Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:29:13Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:29:13Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:29:14 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:14Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "count": 21} 2026/09/05 03:29:14 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:14Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "count": 14} 2026/09/05 03:29:14 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:14Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "count": 3} 2026-09-05T03:29:14Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:29:14Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2be2cf13-bf12-4c92-a7a2-16e164c944e0", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:29:35Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d"} 2026-09-05T03:29:35Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d"} 2026-09-05T03:29:35Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:29:35Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:29:35Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:29:35Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:29:36 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:36Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "count": 21} 2026/09/05 03:29:36 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:36Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "count": 14} 2026/09/05 03:29:36 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:29:36Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "count": 3} 2026-09-05T03:29:36Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:29:36Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "2fec220e-a0a9-4f8b-b251-26c563ecfc0d", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:30:17Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a"} 2026-09-05T03:30:17Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a"} 2026-09-05T03:30:17Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:30:17Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:30:17Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:30:17Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:30:18 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:30:18Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "count": 21} 2026/09/05 03:30:18 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:30:18Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "count": 14} 2026/09/05 03:30:18 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:30:18Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "count": 3} 2026-09-05T03:30:18Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:30:18Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "95ca65c2-2f6c-4d2a-a2c1-563833eca66a", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-09-05T03:31:40Z INFO reconcile {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342"} 2026-09-05T03:31:40Z INFO apply {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342"} 2026-09-05T03:31:40Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).checkDependencies-fm"} 2026-09-05T03:31:40Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).ensureNamespace-fm"} 2026-09-05T03:31:40Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).updateReleases-fm"} 2026-09-05T03:31:40Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm"} 2026/09/05 03:31:41 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:31:41Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered manifests {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "count": 21} 2026/09/05 03:31:41 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:31:41Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered runtimes {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "count": 14} 2026/09/05 03:31:41 Warning: Sorting order is set both in 'kustomization.yaml' ('sortOptions') and in a CLI flag ('--reorder'). Using the kustomization file over the CLI flag. 2026-09-05T03:31:41Z INFO action.github.com/opendatahub-io/trainer-operator/internal/controller.(*trainerActions).renderManifests-fm Rendered imagestreams {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "count": 3} 2026-09-05T03:31:41Z INFO Executing action {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "action": "github.com/opendatahub-io/trainer-operator/internal/controller.NewReconciler.withImmutableFieldRecovery.func11"} 2026-09-05T03:31:41Z ERROR Reconciler error {"controller": "trainer", "controllerGroup": "components.platform.opendatahub.io", "controllerKind": "Trainer", "Trainer": {"name":"default-trainer"}, "namespace": "", "name": "default-trainer", "reconcileID": "d33ad8ce-0572-439c-a573-9f4e41be2342", "error": "provisioning failed: failure deploying resource /trainer-tls-profile: apply failed rbac.authorization.k8s.io/v1, Kind=ClusterRole: unable to patch rbac.authorization.k8s.io/v1, Kind=ClusterRole trainer-tls-profile: clusterroles.rbac.authorization.k8s.io \"trainer-tls-profile\" is forbidden: User \"system:serviceaccount:trainer-operator-system:trainer-operator-controller-manager\" cannot patch resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 client.go:120: Kubernetes events: 0001-01-01T00:00:00Z Normal Scheduled trainer-operator-controller-manager-668cdd7dcf-zzvdv Successfully assigned trainer-operator-system/trainer-operator-controller-manager-668cdd7dcf-zzvdv to ip-10-0-136-28.ec2.internal 0001-01-01T00:00:00Z Normal Scheduled trainer-operator-controller-manager-998d44db8-klgdq Successfully assigned trainer-operator-system/trainer-operator-controller-manager-998d44db8-klgdq to ip-10-0-140-178.ec2.internal 2026-09-05T03:27:21Z Normal SuccessfulCreate trainer-operator-controller-manager-668cdd7dcf Created pod: trainer-operator-controller-manager-668cdd7dcf-zzvdv 2026-09-05T03:27:21Z Normal AddedInterface trainer-operator-controller-manager-668cdd7dcf-zzvdv Add eth0 [10.132.0.18/23] from ovn-kubernetes 2026-09-05T03:27:21Z Normal Pulling trainer-operator-controller-manager-668cdd7dcf-zzvdv Pulling image "quay.io/opendatahub/odh-trainer-operator:odh-stable" 2026-09-05T03:27:21Z Normal ScalingReplicaSet trainer-operator-controller-manager Scaled up replica set trainer-operator-controller-manager-668cdd7dcf from 0 to 1 2026-09-05T03:27:24Z Normal Pulled trainer-operator-controller-manager-668cdd7dcf-zzvdv Successfully pulled image "quay.io/opendatahub/odh-trainer-operator:odh-stable" in 2.403s (2.403s including waiting). Image size: 219739301 bytes. 2026-09-05T03:27:24Z Normal Started trainer-operator-controller-manager-668cdd7dcf-zzvdv Started container manager 2026-09-05T03:27:24Z Normal Created trainer-operator-controller-manager-668cdd7dcf-zzvdv Created container: manager 2026-09-05T03:28:34Z Normal ScalingReplicaSet trainer-operator-controller-manager Scaled up replica set trainer-operator-controller-manager-998d44db8 from 0 to 1 2026-09-05T03:28:34Z Normal SuccessfulCreate trainer-operator-controller-manager-998d44db8 Created pod: trainer-operator-controller-manager-998d44db8-klgdq 2026-09-05T03:28:35Z Normal AddedInterface trainer-operator-controller-manager-998d44db8-klgdq Add eth0 [10.133.0.29/23] from ovn-kubernetes 2026-09-05T03:28:35Z Normal Pulling trainer-operator-controller-manager-998d44db8-klgdq Pulling image "quay.io/opendatahub/odh-trainer-operator@sha256:4d8a5e4ca2aac9f3f2fc71cbb0bf72638f7a5fcdf1ec5c89529aacffbf770734" 2026-09-05T03:28:37Z Normal Pulled trainer-operator-controller-manager-998d44db8-klgdq Successfully pulled image "quay.io/opendatahub/odh-trainer-operator@sha256:4d8a5e4ca2aac9f3f2fc71cbb0bf72638f7a5fcdf1ec5c89529aacffbf770734" in 2.366s (2.366s including waiting). Image size: 219736230 bytes. 2026-09-05T03:28:37Z Normal Started trainer-operator-controller-manager-998d44db8-klgdq Started container manager 2026-09-05T03:28:37Z Normal Created trainer-operator-controller-manager-998d44db8-klgdq Created container: manager 2026-09-05T03:28:48Z Normal SuccessfulDelete trainer-operator-controller-manager-668cdd7dcf Deleted pod: trainer-operator-controller-manager-668cdd7dcf-zzvdv 2026-09-05T03:28:48Z Normal Killing trainer-operator-controller-manager-668cdd7dcf-zzvdv Stopping container manager 2026-09-05T03:28:48Z Normal ScalingReplicaSet trainer-operator-controller-manager Scaled down replica set trainer-operator-controller-manager-668cdd7dcf from 1 to 0 --- FAIL: TestPostUpgradeTrainerStillReady (300.03s) FAIL FAIL github.com/opendatahub-io/trainer-operator/test/e2e/upgrade/post 300.079s FAIL make: *** [Makefile:159: test-e2e-upgrade-post] Error 1