=== events in opendatahub (2026-11-04T02:42:52Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 19m Normal Scheduled pod/maas-controller-84b89c79f6-vsgp7 Successfully assigned opendatahub/maas-controller-84b89c79f6-vsgp7 to ip-10-0-137-185.us-west-2.compute.internal 19m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 to ip-10-0-137-185.us-west-2.compute.internal 8m51s Normal Scheduled pod/test-curl-7241-c0779a Successfully assigned opendatahub/test-curl-7241-c0779a to ip-10-0-137-185.us-west-2.compute.internal 8m43s Normal Scheduled pod/test-curl-7241-8bb509 Successfully assigned opendatahub/test-curl-7241-8bb509 to ip-10-0-137-185.us-west-2.compute.internal 18m Normal Scheduled pod/odh-model-controller-769875db56-fq55l Successfully assigned opendatahub/odh-model-controller-769875db56-fq55l to ip-10-0-129-205.us-west-2.compute.internal 18m Normal Scheduled pod/model-serving-api-75bd89fdfc-z7v7c Successfully assigned opendatahub/model-serving-api-75bd89fdfc-z7v7c to ip-10-0-137-185.us-west-2.compute.internal 19m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-qj2bw Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-qj2bw to ip-10-0-137-185.us-west-2.compute.internal 17m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-s5hpb Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-s5hpb to ip-10-0-137-185.us-west-2.compute.internal 15m Normal Scheduled pod/ai-gateway-controller-7c9cd5f6bf-tjbm7 Successfully assigned opendatahub/ai-gateway-controller-7c9cd5f6bf-tjbm7 to ip-10-0-129-205.us-west-2.compute.internal 18m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-9pzj5 Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-9pzj5 to ip-10-0-137-185.us-west-2.compute.internal 19m Normal Scheduled pod/maas-controller-8b95c9c66-fxvzs Successfully assigned opendatahub/maas-controller-8b95c9c66-fxvzs to ip-10-0-137-185.us-west-2.compute.internal 15m Normal Scheduled pod/ai-gateway-controller-69464886d7-x5xct Successfully assigned opendatahub/ai-gateway-controller-69464886d7-x5xct to ip-10-0-129-205.us-west-2.compute.internal 18m Normal Scheduled pod/kserve-controller-manager-776cc66d47-gfn7w Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-gfn7w to ip-10-0-137-185.us-west-2.compute.internal 19m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-qj2bw 19m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-qj2bw Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 19m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-qj2bw Add eth0 [10.134.0.45/23] from ovn-kubernetes 19m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 19m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-qj2bw Container created 19m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-qj2bw Container started 19m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-qj2bw Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 3.121s (3.121s including waiting). Image size: 195729000 bytes. 19m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-qj2bw_ada51779-4618-4062-8da9-f998285d0501 became leader 19m Normal SuccessfulCreate replicaset/maas-controller-84b89c79f6 Created pod: maas-controller-84b89c79f6-vsgp7 19m Normal AddedInterface pod/maas-controller-8b95c9c66-fxvzs Add eth0 [10.134.0.47/23] from ovn-kubernetes 19m Normal Pulling pod/maas-controller-8b95c9c66-fxvzs Pulling image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" 19m Normal SuccessfulCreate replicaset/maas-controller-8b95c9c66 Created pod: maas-controller-8b95c9c66-fxvzs 19m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-8b95c9c66 from 0 to 1 19m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-84b89c79f6 from 0 to 1 19m Normal AddedInterface pod/maas-controller-84b89c79f6-vsgp7 Add eth0 [10.134.0.48/23] from ovn-kubernetes 19m Normal Pulled pod/maas-controller-84b89c79f6-vsgp7 Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 3.417s (3.417s including waiting). Image size: 200103442 bytes. 19m Normal Pulled pod/maas-controller-8b95c9c66-fxvzs Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 3.863s (3.863s including waiting). Image size: 200103442 bytes. 19m Normal Created pod/maas-controller-8b95c9c66-fxvzs Container created 19m Normal Started pod/maas-controller-8b95c9c66-fxvzs Container started 19m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-8b95c9c66-fxvzs_04de7b79-2bf4-4fc1-ba21-d9b354346f2d became leader 19m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 19m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 19m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found 19m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 Add eth0 [10.134.0.49/23] from ovn-kubernetes 19m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 19m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 18m Normal Killing pod/maas-controller-8b95c9c66-fxvzs Stopping container manager 18m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-8b95c9c66 from 1 to 0 18m Normal SuccessfulDelete replicaset/maas-controller-8b95c9c66 Deleted pod: maas-controller-8b95c9c66-fxvzs 18m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 14.646s (14.646s including waiting). Image size: 1089502687 bytes. 18m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 Container created 18m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-7fpg9 Container started 18m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-9pzj5 18m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-gfn7w 18m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 18m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 18m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 18m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-z7v7c 18m Normal AddedInterface pod/odh-model-controller-769875db56-fq55l Add eth0 [10.133.0.27/23] from ovn-kubernetes 18m Normal Pulling pod/odh-model-controller-769875db56-fq55l Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 18m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-gfn7w Add eth0 [10.134.0.50/23] from ovn-kubernetes 18m Normal Pulling pod/kserve-controller-manager-776cc66d47-gfn7w Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 18m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-9pzj5 Add eth0 [10.134.0.51/23] from ovn-kubernetes 18m Normal AddedInterface pod/model-serving-api-75bd89fdfc-z7v7c Add eth0 [10.134.0.52/23] from ovn-kubernetes 18m Normal Pulling pod/model-serving-api-75bd89fdfc-z7v7c Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 18m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 18m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-fq55l 18m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-9pzj5 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 18m Normal Started pod/model-serving-api-75bd89fdfc-z7v7c Container started 18m Normal Created pod/model-serving-api-75bd89fdfc-z7v7c Container created 18m Normal Pulled pod/model-serving-api-75bd89fdfc-z7v7c Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.452s (2.452s including waiting). Image size: 184925761 bytes. 18m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-9pzj5 Container started 18m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-9pzj5 Container created 18m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-9pzj5 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.516s (2.516s including waiting). Image size: 235100981 bytes. 18m Normal Pulled pod/odh-model-controller-769875db56-fq55l Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.213s (3.213s including waiting). Image size: 236755567 bytes. 18m Normal Created pod/odh-model-controller-769875db56-fq55l Container created 18m Normal Started pod/odh-model-controller-769875db56-fq55l Container started 18m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-84b89c79f6-vsgp7_77971f59-8911-426f-8080-e9e18b44489e became leader 18m Normal Created pod/kserve-controller-manager-776cc66d47-gfn7w Container created 18m Normal Started pod/kserve-controller-manager-776cc66d47-gfn7w Container started 18m Normal Pulled pod/kserve-controller-manager-776cc66d47-gfn7w Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 4.95s (4.95s including waiting). Image size: 247676581 bytes. 18m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-fq55l_5779a8f0-2b40-46f7-8bef-148f9c8207f5 became leader 18m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-9pzj5_6c814bc4-9499-41bc-8323-051c604d0eed became leader 18m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-gfn7w_fcfb5220-dbde-45d5-b062-fb2007252542 became leader 17m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-s5hpb 17m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 17m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-s5hpb Add eth0 [10.134.0.54/23] from ovn-kubernetes 17m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-s5hpb Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 17m Normal Started pod/llmisvc-controller-manager-6466c554d8-s5hpb Container started 17m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-s5hpb Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 924ms (924ms including waiting). Image size: 235100981 bytes. 17m Normal Created pod/llmisvc-controller-manager-6466c554d8-s5hpb Container created 16m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-9pzj5 Stopping container manager 16m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-9pzj5 16m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 16m Normal Pulling pod/maas-controller-84b89c79f6-vsgp7 Pulling image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" 16m Normal Pulled pod/maas-controller-84b89c79f6-vsgp7 Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 1.062s (1.062s including waiting). Image size: 200103442 bytes. 16m Normal Started pod/maas-controller-84b89c79f6-vsgp7 Container started 16m Normal Created pod/maas-controller-84b89c79f6-vsgp7 Container created 16m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-s5hpb_d3acdf54-7eda-4b65-8766-6d81ac999db4 became leader 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-84b89c79f6-vsgp7_f672dcb1-d1ef-421f-b6ff-f03e2d300e11 became leader 15m Normal SuccessfulCreate replicaset/ai-gateway-controller-69464886d7 Created pod: ai-gateway-controller-69464886d7-x5xct 15m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-69464886d7 from 0 to 1 15m Normal AddedInterface pod/ai-gateway-controller-69464886d7-x5xct Add eth0 [10.133.0.34/23] from ovn-kubernetes 15m Normal Pulling pod/ai-gateway-controller-69464886d7-x5xct Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" 15m Normal Started pod/ai-gateway-controller-69464886d7-x5xct Container started 15m Normal Pulled pod/ai-gateway-controller-69464886d7-x5xct Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" in 6.195s (6.195s including waiting). Image size: 184136095 bytes. 15m Normal Created pod/ai-gateway-controller-69464886d7-x5xct Container created 15m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-69464886d7-x5xct_14259a05-34df-40f0-9eae-9c4c0bbcf1ef became leader 15m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-7c9cd5f6bf from 0 to 1 15m Normal SuccessfulCreate replicaset/ai-gateway-controller-7c9cd5f6bf Created pod: ai-gateway-controller-7c9cd5f6bf-tjbm7 15m Normal AddedInterface pod/ai-gateway-controller-7c9cd5f6bf-tjbm7 Add eth0 [10.133.0.36/23] from ovn-kubernetes 15m Normal Pulling pod/ai-gateway-controller-7c9cd5f6bf-tjbm7 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" 15m Normal Created pod/ai-gateway-controller-7c9cd5f6bf-tjbm7 Container created 15m Normal Started pod/ai-gateway-controller-7c9cd5f6bf-tjbm7 Container started 15m Normal Pulled pod/ai-gateway-controller-7c9cd5f6bf-tjbm7 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" in 951ms (951ms including waiting). Image size: 184136095 bytes. 15m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-69464886d7 from 1 to 0 15m Normal SuccessfulDelete replicaset/ai-gateway-controller-69464886d7 Deleted pod: ai-gateway-controller-69464886d7-x5xct 15m Normal Killing pod/ai-gateway-controller-69464886d7-x5xct Stopping container manager 14m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-7c9cd5f6bf-tjbm7_60b8730d-6521-44ea-8504-b83d4b914e2b became leader 8m51s Normal AddedInterface pod/test-curl-7241-c0779a Add eth0 [10.134.0.69/23] from ovn-kubernetes 8m51s Normal Pulling pod/test-curl-7241-c0779a Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 8m47s Normal Created pod/test-curl-7241-c0779a Container created 8m47s Normal Started pod/test-curl-7241-c0779a Container started 8m47s Normal Pulled pod/test-curl-7241-c0779a Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 4.441s (4.441s including waiting). Image size: 106100986 bytes. 8m44s Normal Killing pod/test-curl-7241-c0779a Stopping container test-curl-7241-c0779a 8m43s Normal Pulled pod/test-curl-7241-8bb509 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 8m43s Normal AddedInterface pod/test-curl-7241-8bb509 Add eth0 [10.134.0.70/23] from ovn-kubernetes 8m43s Normal Created pod/test-curl-7241-8bb509 Container created 8m43s Normal Started pod/test-curl-7241-8bb509 Container started 8m41s Normal Killing pod/test-curl-7241-8bb509 Stopping container test-curl-7241-8bb509